Deeplight : Robust & Unobtrusive Real-time Screen-Camera Communication for Real-World Displays

Vu Tran, Gihan Jayatilaka, Ashwin Ashok and Archan Misra, 2021, April. Deeplight : Robust & Unobtrusive Real-time Screen-Camera Communication for Real-World Displays. In 2021 20th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN) (). IEEE.

[Preprint PDF]

Download

Code
ScreenNet weights
LightNet weights
Dataset for ScreenNet (Only a part of dataset is here)

Overview diagram

System architecture

Cite

@InProceedings{deeplight2021,
	author = {Tran, Vu and Jayatilaka, Gihan and Ashok, Ashwin and Misra, Archan},
	title = {Deeplight : Robust & Unobtrusive Real-time Screen-Camera Communication for Real-World Displays},
	booktitle = {ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN)},
	month = {May},
	year = {2021}
}

Supplementary material

1. The images used to train DeepLight Screen Extractor [Indoor scenes, Outdoor scenes]
2. The representative images of videos used in the user study
3. The representative images of videos used to train DeepLight Decoder
4. The representative images of videos used in the experiment
5. A video showing DeepLight operation in real-time (deeplight_demo.mov).
In this demo, the iPhone captures video frames of a transmitter screen and process them locally. After the user presses the start button, the DeepLight pipe-line runs for 32 frames and continuously update the results while processing the frames. The screen detector runs only once (the first frame) for 32 frames. The reponse time of DeepLight shows that it supports real-time processing.