Title: Resolution and Scale-aware Computer Vision
Advisor(s): Luis Ceze
Supervisory committee: Luis Ceze (Chair), Arka Majumdar (GSR, Electrical and Computer Engineering), Xi Wang, Zachary Tatlock
Abstract:
Modern CNN architectures are effectively resolution-independent from a shape perspective: models trained at a resolution can perform inference at an arbitrary resolution, where the possible benefits of doing so depend on the scale of the inference input image. Despite this property, however, most CNN pipelines still perform inference at a fixed resolution. In the proposed project, we aim to show that from the perspectives of storage efficiency, compute utilization, and model accuracy, it is beneficial to perform inference at dynamic resolutions for computer vision models. From the storage perspective, we introduce dynamism that varies the amount of data read for inference in a resolution-dependent manner. For optimal compute utilization, we leverage automatic code generation and tuning to produce program variants that maximize hardware utilization given different input resolutions. Finally, we evaluate an alternative model pipeline that uses a lightweight model to select the optimal resolution for inference first to minimize compute utilization while improving model accuracy. We will evaluate the combination of techniques from an efficiency and accuracy perspective, aiming to design a system that is pareto-optimal in terms of both accuracy vs. bytes read per image and accuracy vs. FLOPs per image.