Odessa: Enabling Interactive Perception Applications on Mobile Devices

Motivation

Interactive perception is a holistic approach towards autonomous exploration. Interactive perception enabled applications require crisp response. Any kind of latency or delay with sensory feedback will degrade user experience. The integral part of this application requires continuous manipulation and processing of the sensor data outputs to be accurate. A less accurate output will also degrade user experience. Lastly, data processing and machine learning algorithms that help the device track objects and extract features are compute intensive. These requirements cannot be satisfied using current generation smart devices due to the devices having limited computation and battery power. This is where a lightweight runtime named Odessa is introduced to dynamically offload compute intensive requirements by adaptively making offloading and parallelism decisions for mobile interactive perception applications.

Background

• The paper introduces two metrics to measure the accuracy and responsiveness of the interactive perception system:

o      Makespan: is the time taken to execute all stages of a data flow graph for a single frame. This measures the responsiveness of the application.

o       Throughput: is the rate at which the frames are processed. This measures the accuracy of the application.

o      The system strives to maintain high throughput and low makespan.

o      Reduce makespan: offloading computationally intensive stages to the cloud reduces makespan.

o      Increase throughput: different stages of application process different frames in parallel using pipelining.  This pipelining technique increases throughput.

•       Sprout: a distributed stream processing engine that executes parallel applications. Odessa is built on Sprout that takes advantage of the engine’s continuous processing of high rate streaming data.
•       The paper focuses on three different computer-vision applications – face recognition, object and pose recognition and gesture recognition to study and evaluate the performance of Odessa runtime.

·        Main Points

•       The paper makes three contributions towards the development of the paper:

o      The paper, though extensive experiments, provides an understanding of the factors that help the system make offloading and parallelism decisions.

o      Discusses the design of Odessa with its goal of simultaneously improving accuracy and responsiveness by exploring offloading decisions to make and adapting processing of frames in parallel

o      Finally, the paper provides an insight on how Odessa manages to show positive results with 3 times improvement in performance over an idealized offline configuration computation and performance profiling.

•       Odessa adaptively exploits offloading, pipelining and data-parallelism to achieve high accuracy and fast responsiveness as discusses before. Additionally, it also reacts quickly to the scene complexity, compute resource availability and network performance.
•       Odessa uses a greedy algorithm that acquires information from a lightweight application profiler that estimates the bottleneck in current configuration. The application profiler maintains a performance profile of the running application. The profiler extracts information regarding each processed frame and makes this information available to the Decision Engine.
•       The decision engine executes two threads that use information from the profiler to help the engine decide on what stages to adaptively offload and frames to be distributed and run in parallel using pipelining and data-parallelism techniques. The first thread of this engine manages offloading and data parallelism. The second thread of the engine manages pipeline parallelism.
•       Ideally, application fidelity helps decision engine to decide the level of data-parallelism to be achieved, however the paper acknowledges that this has been left for future work.

Closing Thoughts

•       In summary, Odessa makes offloading decisions in an adaptive way.
•       Although, data-parallelism help increase application throughput, but the level of parallelism cannot be determined a priori.
•       Additionally, a static choice of pipeline parallelism can cause sub-optimal performance and this choice should be left to decision engine as it has access to information delivered by the lightweight profiler.

Trade-offs

•       The paper gives a very detailed explanation on the design and the evaluation of runtime Odessa. Keeping mobile device as the focus, it does not explain what part of mobile application needs to be resource ready. Moreover, the paper does not recognize the plumbing needed on the applications running on the mobile device.
•       Even though Odessa built on Spout, we do not know how Odessa will perform with other parallel applications other than the applications mentioned in this paper.