DeepSense 6G

A Large-Scale Real-World Multi-Modal Sensing and Communication Dataset for
6G Deep Learning Research

0 +

 Downloads Since Release in March 2022

News

15 April 2024

DeepSense 6G Workshop 2024!

We are pleased to announce the 1st DeepSense 6G Workshop. For those interested in exploring the capabilities of the DeepSense 6G dataset and learning how to effectively utilize it, please register and join us on one of the following dates:

Session 1: May 2, 2024, 7:00 PM – 9:30 PM (PST) [10a-12:30p Beijing Time]

Session 2: May 8, 2024, 9:00 AM – 11:30 AM (PST)

Session 3: May 22, 2024, 5:00 AM: 7:30 AM (PST) [2p-4:30p CET]

Click here to Register!

testbed_pic_edited2

15 April 2024

Scenarios 42-44 are available!

The first real-world ISAC scenarios with 3D Lidar and vision modalities!

Fig_testbed

21 March 2024

Scenario 41 is now available!

The first real-world distributed sensing scenario!

0287F76C-4DAE-4435-ADFF-7D9682592BEF

21 March 2024

Scenario 35 is now available!

Realistic multi-modal scenario (gNodeB at ~11m height)!

ranking_v2

21 December 2023

Congratulations to the Winners of the 2023 challenge!

Final ranking of the 2023 challenge is available.

ranking_v2

21 December 2023

Congratulations to the Winners of the 2023 challenge!

Final ranking of the 2023 challenge is available.

testbedv2

21 December 2023

Multi-Modal V2V Beam Prediction Task!

The 2023 V2V ML task is now available!

01 July 2023

The First Multi Modal Sensing and Comm. V2V Dataset!

Scenarios 36-39 are now available!

Screenshot 2022-12-11 150746

10 December 2022

Congratulations to the Winners!

Final ranking of the 2022 challenge is available.

32

10 December 2022 

Multi-Modal Beam Prediction! 

New tasks and datasets are available.

image_4610

10 December 2022

DeepSense Scenarios Dataset! 

Many scenario datasets are now available!

DS TB1 copy

03 October 2022

DeepSense 6G Testbeds!

Descriptions of testbeds 2, 3, and 5 are now available.

problem_statement

20 June 2022

DeepSense-ITU ML Challenge

Registration is now open!

DSC00146

19 May 2022

New ML tasks are available!

Radar, LiDAR, position-based beam and blockage prediction.

DSC00204_cover_v2

19 May 2022

New scenarios are released!

Scenarios 6 – 9, 17 – 22 , 24 – 29 and 30 are now available.

DSC00165

17 April 2022

First ML task data is available!

Vision-aided beam prediction dataset is released.

ViWi_video_animation_shot1_4k 38 (2)

10 April 2022

New scenarios are released!

Scenarios 3, 4, and 5 are now available.

DSC00204_cover_v2

19 May 2022

New scenarios are released!

Scenarios 6 – 9, 17 – 22 , 24 – 29 and 30 are now available.

DSC00165

17 April 2022

First ML task data is available!

Vision-aided beam prediction dataset is released.

Image from iOS (4)

22 March 2022

First two scenarios released! 

Scenarios 1 and 2 are now available.

DS TB1 copy

22 March 2022

DeepSense website is live!

The datasets will be released very soon. 

ViWi_video_animation_shot1_4k 38 (2)

10 April 2022

New scenarios are released!

Scenarios 3, 4, and 5 are now available.

Image from iOS (4)

22 March 2022

First two scenarios released! 

Scenarios 1 and 2 are now available.

Reproducing the Research Results

What is DeepSense 6G?

DeepSense 6G is a real-world multi-modal dataset that comprises coexisting multi-modal sensing and communication data, such as mmWave wireless communication, Camera, GPS data, LiDAR, and Radar, collected in realistic wireless environments 

  • World’s first large-scale real-world sensing-communication dataset 
  • 1Million+ multi-modal sensing-communication data samples 
  • Modalities include mmWave comm., GPS, Image, LiDAR and Radar
  • 30+ scenarios targeting various applications 
  • Several indoor/outdoor locations with high diversity 
  • Data collected at different times of day and weather conditions
  • Tens of thousands of manually and auto labeled data 

DeepSense Scenarios

The DeepSense 6G dataset is offered in different scenarios. The different scenarios cover a diverse vehicular, drone, indoor/outdoor scenarios. Each scenario consists of co-existing multi-modal sensing and communication data

Key Highlights:
  • 30+ scenarios collected over several locations
  • Both indoor and outdoor scenarios
  • Locations range from parking lots to busy downtown streets
  • Scenarios comprise  data collected at different times of the day
  • Scenarios for drones, vehicles, RIS, human, robots, and more … 
  • Scenarios are designed to enable different novel applications

For more details, please click on the link below. 

Applications and ML Tasks

DeepSense 6G enables a wide range of applications in communication, sensing, and localization. 

To encourage the development of machine learning solutions for these applications and motivate benchmarking and comparisons, we define specific machine learning task. 

For each DeepSense 6G-based task, we provide (i) Task definition and formulation, (ii) diverse real-world task-specific development dataset, and (iii) benchmarking scripts. For more details, please click on the link below

License

In order to use the DeepSense dataset/scripts or any (modified) part of them, please cite:

A. Alkhateeb, G. Charan, T. Osman, A. Hredzak, J. Morais, U. Demirhan, and N. Srinivas, “DeepSense 6G: A Large-Scale Real-World Multi-Modal Sensing and Communication Datasets,” IEEE Communications Magazine, 2023.

@Article{DeepSense,
author={Alkhateeb, Ahmed and Charan, Gouranga and Osman, Tawfik and Hredzak, Andrew and Morais, Joao and Demirhan, Umut and Srinivas, Nikhil},
title={DeepSense 6G: A Large-Scale Real-World Multi-Modal Sensing and Communication Dataset},
journal={IEEE Communications Magazine},
year={2023},
publisher={IEEE}}

Join the DeepSense mailing list to get the recent updates