Tiny-ML toolchain for ultra-constrained processors
PhD project information
The objective of this PhD is to develop an innovative AI mapping toolchain capable of:
- Architecting artificial neural networks at very low bit depth (i. e., down to 1 bit activation/weight data width vs. 32 bit floating point data width)
- training resulting models on selected datasets representing use cases
- optimizing their topologies with multi target objectives (e.g. memory, energy, computation).
The aim for implementing the software in a hardware is the ultra-low power DSP (code name STred – reduced energy DSP) processor unit designed by ST featuring very constrained memory and computational resources.
Potential commercial and industrial applications of this PhD is in the development of Distributed, Intelligent, Self-aware IoT Applications. For instance, this processor-software solution has applications in audio (e.g. ultrasound event detection) and vibration processing (e.g. anomaly detection) for predictive maintenance and within a power envelop comparable as order of magnitude to the energy consumption of the sensor alone (i.e. µW). Furthermore, interoperability with existing ST AI tools for automatic mapping and deployment of pre trained neural networks shall be in the scope of the activity.
Distributed AI is fast growing field requiring fast innovation to surf its wave. This can be enabled by offering neural network inference up to ultra-constrained devices within a coherent, productive toolchain and an end-to-end eco-system. Indeed, machine learning inference on the edge is an increasingly attractive prospect due to its potential for increasing energy efficiency privacy, responsiveness, and autonomy of edge devices.
In this PhD is foreseen that development of the innovative AI toolchain will support our development, optimization and mapping of Tiny Machine Learning applications on resource constraints processors on the edge. In particular the tools should: a) Architect artificial neural networks at very low bit depth (i. e., down to 1 bit activation/weight data width); b) training resulting models on selected datasets representing use cases, optimized for the deployment scenarios; c) optimizing their topologies with multi target objectives (e.g. memory, energy, computation). In all cases these tools and techniques should be designed to be interoperable and demonstrated with existing AI toolchain (e.g. ST and off the shelf).
The outcomes of the PhD will be used by ST to develop of several new technologies that will address the following research questions: how to build an optimal neural network topology for such constrained processors? How to optimize it considering available computing resources? What type of additional requirements will be required to existing processors? How much more productivity will it be offered to AI embedded developers? How to specify the requirements of learning phases when such extremely constrained devices are the final target?
The outcome of the thesis will be integrated into the STMicroelectronics AI toolchain with a novel module for supporting ultra-low power nodes for TinyML applications. The release of the developed toolchain and test cases used for validation and demonstration, will be done, not only internally to ST divisions, but also together with the hw-platform to ST customers, ensuring a strong industrial impact of the research
Regarding the market, there are 250 billion microcontrollers in the world today. 28.1 billion units were sold in 2018 alone, and IC Insights forecasts annual shipment volume to grow to 38.2 billion by 2023. TinyML is targeting all of them as described at https://venturebeat.com/2020/01/11/why-tinyml-is-a-giant-opportunity/ All market segments are affected including IoT, Industrial, Automotive etc.
This PhD will support ST with the creation of a new processor integrated within the sensor. We have a first silicon available. However, details related to this are not yet public.
Current solution is https://www.st.com/en/mems-and-sensors/lsm6dsox.html with a classic machine learning core implementing decision trees, however this will be further improved with more complicated solutions. Main envisioned business sectors are: Human Activity recognition, event/anomaly detection for industrial application, smartphones.
- Industrial partner: STMicroelectronics
- Academic/research partner: Politecnico di Milano
- Number of available PhD positions: 1
- Duration: 3 years
- Innovation Focus Areas(s): Digital Industry, Digital Tech
- DTC location: Milano (Trento)
- This PhD will be funded by EIT Digital and STMicroelectronics
PhD thesis motivation and innovation valorisation
Advancements in ultra-low power processors and sensors as well as in artificial intelligence are paving the way towards increasingly autonomous intelligent Cyber-Physical Systems. A CPS environment at the deep edge is equipped with resource-constrained devices. In turn it embodies a sensor node which executes CPS edge applications exploiting sensing, computing, communication and actuation. Other than classic CPS deployments, where a centralized CPS cyber platform was the central point of data collection and processing, intelligent distributed CPS environments aim to rely (mostly) on computation capabilities implemented very close to sensors (or within same package) and actuators in order to decentralize intelligence and minimize cloud connectivity/latency issues and privacy concerns [1-3].
The Intelligent CPS advancements will allow the implementation of break-through use cases at unprecedented ultra-low power consumption. For example in manufacturing, a plant can be highly automated and shared by multiple tenants who utilize machinery from trusted third-party vendors. In this context, devices implementing tiny machine-learning applications can be used to monitor the manufacturing process and dealing with unknown behaviors or unknown defective products not foreseen at product qualification time.
 S. Higginbotham, "Machine learning on the edge" in IEEE Spectrum, vol. 57, no. 1, pp. 20-20, Jan. 2020.
“Engineers are Pushing Machine Learning to the World’s Humblest Microprocessors” IEEE Spectrum Online version.
The main novelty of this PhD consists of application of deep-edge neural networks in intelligent CPS systems supported by distributed AI on CPS ultra-constrained processors in a sensor node. Such processors are resource constrained (typically, KB of memory, few MHz clock frequencies, µW power budget, dedicated binary/low bit depth instructions).
The innovative solution will be built based on the following three pillars:
Intelligence and autonomy. Considering the individual CPS sensor node, applications that leverage decentralized AI involve a range of components such as sensors, micro controllers, gateways; moreover distributed intelligence must be provided through the collaboration of such assets, without continuously relying on the cloud connectivity and computing resources. On one hand, AI models must be optimized so that severely resource-constrained CPS devices will be able to run them even though at the deep edge rich compute capabilities may not be present. On the other hand, the CPS environment must allow for re-deployment of updated, re-trained AI models.
Constrained communication and computation: CPS environments, e.g. a manufacturing plant, must (at least temporarily) be able to operate without access to public networks, e.g., due to remoteness, or outages. Additionally, privacy concerns might prohibit the exchange of critical data through public networks. Furthermore, the distribution of computation resources must be carefully balanced.
Distributed, Intelligent, Self-aware IoT Applications: Nodes are self-aware and can learn, e.g., to detect and autonomously mitigate failures. This makes autonomous un supervised behavior within the distributed system possible. Distributed AI is enabled by components offering neural inferences on constrained devices.
Expected academic outcomes
- Techniques and tools for supporting the development, optimization and mapping of TinyML applications on resource-constrained-processors
- Knowledge and the answer to the key research questions of the PhD topic will be disseminated through several MSc level courses at POLIMI, by preparing ad Hoc seminars but also by supervising MSc students’ thesis or course projects.
- Publication of scientific papers on international journals and conferences related to architecture, compiler and exemplary ANN(s) are envisioned to ensure a significant scientific impact of the research. Quantitative expected outcomes are at least 3 top-level conferences and 2 journal submissions.
Concrete innovations expected as the outcome of the proposal
The expected innovation outcome of the proposal relies in:
- The development of an innovative software module for supporting ultra-low power nodes for Tiny Machine Learning applications. This will contribute to the improvement of the STMicroelectronics AI toolchains business line.
- The development of test cases/pilots to test the developed technology. It is expected that the pilot testing will include scenarios where it is proved in industrial environments.
- The validation of the developed toolchain and test cases will be done with the support of software and hardware ST divisions. The validation will be done using the ST hardware-platform for customers, ensuring a strong industrial impact of the PhD outcomes.
Moreover, given the research&innovation nature of the PhD, we cannot exclude the possibility to have also a patent (or a set of patents) as possible outcome of this work especially on the activities that are on the border between HW and SW.
Expected impact of the PhD outcomes with respect to their business line
The Phd outcomes not only will strength the collaboration between ST and PoliMI feeding the local innovation ecosystem in the Milano area, but it is expected also to have a direct impact on the business perspective by improving the presence of STMicroelectronics in the fast growing field of distributed Tiny AI, and ultra-low power sensors. Indeed, offering neural network inference up to ultra-constrained devices within a coherent and productive toolchain ecosystem will be a strong plus for business opportunities exploitation.
- Regarding new services and attraction of new costumers: ST will serve more productively the mass market community (https://community.st.com/s/) investing on AI for embedded devices. It is hard to envision in this moment possible new customers, for sure the new technology will improve the ST position in the business sectors mentioned before, i.e. Human Activity recognition, event/anomaly detection for industrial application, smartphones
- Regarding the company having a leading position over competitors: Historically, ST is one of the market leaders for hardware sensors. However the market scenario of smart sensors its more complicated, and the possibility to have an innovative combination of hw/sw solutions for TinyML applications will permit to compete better against competitors
- Regarding the company leading in the creation of new standards: ST is active and founder of the TinyML Foundation since 2019 by sharing its activities in this research and market context. Moreover, ST and the proposer of the PhD topic from ST, already contributed to the TinyMLPerf community. TinyMLPerf is the effort within the MLPerf related to tiny Machine Learning. It started since late 2019 and ST was part of it since the beginning. MLPerf was founded in February, 2018 as a collaboration of companies and researchers with the mission of fair and useful benchmarking of ML hardware, software, and services. ST is set to further contribute in this direction.
- Regarding new portfolio of services/products will expand to new markets: As mentioned previously, The portfolio of product will be increased by bringing even more intelligence within the sensor node. A new product is under design. Also an already available senso such as https://www.st.com/en/mems-and-sensors/lsm6dsox.htm will be a candidate to be further developed to bring more modern AI applications. Moreover, The toolchain developed in during the PhD period, will allow seamless integration of cloud services with distributed AI. ST has cloud solutions distributed such as:
PhD thesis time-line and milestones
Hands on trainings on existing ST constrained processor toolchain and ST code generation toolchain and exemplary ANN applications completed.
Initial contributions on existing ST code generation toolchain.
First submission of achieved results / the proposed approach to an international workshop/conference.
International mobility plan
- France: STMicroelectronics (Grenoble)
- Switzerland: ETH-Zurich
- Portugal: University of Porto
- Cyprus: KIOS Research and Innovation Center of Excellence
- University of Cyprus
Will the PhD Student do the Business Development Experience at Industrial Partner premises?