NICTA has posted this research strategy for the purposes of obtaining feedback from ESA companies about NICTA research direction and facilitating collaboration.

OVERVIEW

At NICTA, five research areas of strategic importance are identified that leverage our core strengths and are aligned to some of the opportunities that emerge from the identified trends in the software and hardware design steps of the engineering process.

1. Systems and Software Engineering

Embedded systems are growing in complexity and are among the most-complex artificial systems that exist. For large systems, such as automotive vehicles, the architecture of the system dominates its engineering. An architecture that poorly fits as a solution to a control system problem will result in a poor control system regardless of the excellence of the subsequent steps in the product engineering process.

For small systems, the architecture is often fairly obvious. In many cases, such systems are over-engineered by a significant factor. While it is possible to quantify how well an architectural solution “fits” a problem, such analysis is rarely performed in practice. The consequences of a poor architectural fit can include increased design time, verification and production cost, increased power consumption and less assurance that the product will be fit for service in all operating conditions.

There exist many architectural solutions to a given problem. An overarching principle driving embedded systems architectural research lies in the way designers arrive at a particular solution. One school of thought is essentially top-down and involves quantitatively selecting an architecture that best fits the mathematical models that are derived from the problem specification. This approach results in an efficient solution with a high degree of fit between the architecture and the problem. It is a principle of this approach to systems design that the verification process is an integral part of each step of the engineering process that results in the validation of a system with regard to its specification without additional work. The challenge is deriving the mathematical model from the specification, and then separating out the control model and timing such that the embedded control system can be defined. Parameters that define the architecture, functional processing and synchronisation, bandwidth, and memory subsystems emerge from this control model. The constraints of the specification are then carried for into the design. Success with this approach comes from an excellent requirement specification (that is use-case driven), followed by a set of testable executable specifications derived from the requirements, quantitative assessment of candidate architectures, mapping to implementation technologies such as software, electronics and mechanics.

 Another school of thought takes a bottom-up approach and involves the composition of ready-made sub-system components (of both hardware and software) in such a way that the system level requirements are satisfied. This design by composition approach attempts to facilitate the rapid assembly of embedded systems with reduced unit-level testing, while meeting the requirements of the system specification. In this approach, the underlying architecture emerges from the composition of the components that make up the architectural solution. The fit of the solution to the problem is less important than the ability to re-use components. Another approach could explore a combination of these two approaches – one where the model of the control system is used to select the architectural solution with the best fit – with the constraints of the composition of the underlying architectural alternatives. The overarching principle yields a general embedded systems technology platform that is empirical in nature – science driving engineering.

2. Secure, Reliable, Trustworthy Embedded Software

Software accounts for a significant (frequently the dominant) fraction of the complexity of embedded systems. Large, complex, feature-rich software systems are notoriously difficult and expensive to construct reliably, safely and securely. The ERTOS group will develop embedded systems software platforms, and associated technologies, which enable the construction of secure, safe, and reliable complex embedded systems. Our approach revolves around trustworthy microkernel-based operating systems, tools for construction and analysis of component-based systems constructed upon microkernels, and the strong enforcement of functional and temporal isolation boundaries between components, enabling systems composed of both highly-trustworthy software and untrusted software. The approach will enable vendors to supply embedded systems where critical functionality is secure, safe, and reliable, by allowing them to focus their high-assurance software-development strategies on the critical functionality. Non-critical and untrusted software can be incorporated, and the platform ensures that untrusted software is strongly isolated except for carefully-architected and -controlled interaction between software at differing levels of trust. Embedded-systems vendors can deliver systems with complex features sets possessing differing levels of trust, or systems consolidating previously separate systems, and still have ultimate trust in the functionality and temporal validity of critical components of the system, while reducing their development costs for untrusted (or third party) software via reduced assurance development or outsourcing.

Several technologies are required to succeed in our approach. A truly trustworthy foundation is required to enforce the isolation guarantees required. NICTA has developed a completely mathematically verified implementation of a microkernel to serve as a trustworthy foundation of embedded systems. This is complemented by techniques to identify the temporal behaviour of the microkernel. We will develop operating-system components that provide system services to higher-level software layers, such as a secure user-interface components. Some of these components will possess the same rigorous guarantees as the underlying kernel. We will develop a system-composition framework which enables vendors to architect systems that take advantage of the mathematically proven isolation guarantees, in a mathematically provable way, without the vendors requiring formal methods or real-time analysis experts. While isolation in the presence of failure is critical to our approach, we also expect to develop analysis tools for our platform which aids in the development of correct systems and minimise the reliance on the strong isolation guarantees.

The objective is to make ERTOS's platform and surrounding technologies the de-facto standard for vendors of embedded systems that require security, reliability and real-time guarantees for some, but not necessarily all parts of the system, as well as for vendors who outsource components and need a secure, safe, and reliable approach to combining (software or hardware+software) components produced by third-party vendors.  

3. Dynamic Scene Understanding on Visual Sensor Networks

This activity will develop systems for object identification and tracking in real time using intelligent networks of smart cameras. In the future, there will be millions of cameras deployed as hierarchies of dynamically reconfigurable visual sensor networks. These networks of cameras will work together to perform dynamic scene analysis with distributed algorithms. They will be able to track objects between scenes. Furthermore, sensors will themselves move about freely in space. Cameras will be placed in cars, and on pedestrians (like smart eyewear with real time augmented reality or visual sensors for bionic eye processing to assist people with vision impairments). The other trend that we acknowledge here is that “smarts” are migrating out to the sensors. Smart sensors will measure the environment and be able to store the measurements, process the data, store the results and transmit some or all of the data to other sensors and back through the network for processing. It is with these trends in mind, that we create a theme plan for embedded dynamic scene analysis. In doing so, we combine the unique mix of NICTA’s computer vision capability, with sensor networking and the making sense of data research theme – to create solutions that leverage the skills to create competitive advantage.

The need to extract real-time information from dynamic scenes is a unifying requirement of a broad range of NICTA's projects that require computer vision. Rather than a single camera, or multiple cameras – we need visual sensor networks that can, in real time give information about self-motion, view-points of other cameras that are linked in the scene, the structure of the scene, and the motion of any objects in the scene. At the same time, the sensor network should identify certain key objects in the scene. Indeed these requirements are key to a broad range of possible sensing technologies in Intelligent Transport, for both in car devices and road infrastructure, in video surveillance, and in low-vision assistive devices.

To underpin this requirement, we propose to develop embedded technologies that incorporate smart visual sensor networks for the analysis of dynamic scenes. These visual sensors make use of a mix of embedded systems to facilitate real-time analysis, and include NICTA algorithms for camera, and scene structure and object motion recovery. It will also identify a class of objects in the scene.

The basic mode of operation of the camera will be to given trained descriptors of a set of objects. These objects will be recognised and tracked whenever they appear in the visual scene, and the segmented objects, their respective positions and motions as well as that of the cameras will be output via wireless network from the camera. If there are multiple cameras in a scene, these will automatically interface to each other (to form a visual sensor network), and work together to improve rates of object recognition, and quality of positional and motion data.

Networks of cameras might be controlled via a network controller that performs operations equivalent to soft hand-over in a mobile communications system – enabling objects to be tracked across groups of sensor networks. This creates a scalable system.

To build such a system will require research in embedding computer vision algorithms on hardware, on real-time and parallel algorithms for detection/recognition/tracking of objects and scene structure and motion recovery.

4. Wireless Systems and Sensors on-a-Chip

The goal of this research is to develop technologies for ultra low cost, low power, smart sensor systems on a chip. Areas of interest include 60GHz Wireless Personal Area Networking in CMOS, Body Area Networking, Communications links with implanted devices, and energy harvesting sensors with wake-up radios. This activity exploits NICTAs expertise in RFCMOS microelectronics design for integrated wireless communications. Projects would focus on developing CMOS chips for communicating between consumer electronics goods at multi-gigabit per second (for the WirelessHD, 802.11VHT and 802.15.3C standards), for communicating around the body (as per the 802.15.6 MBAN standard), or for low data rate systems like proprietary links for implanted devices and Zigbee for sensor networking. In each case, the wireless sub-system is integrated with other functionality (e.g. temperature, acceleration, location sensors, baseband/MAC layers, applications processors etc) to create a complete sub-system on a chip solution. A long term goal might be to create smart dust with MEMs based energy harvesting to power on-chip sensors and radios. Intermittent communication bursts would be used (in a wake-up radio architecture) to get data off the chip to regional processing nodes.  

5. Co-Design of Dynamically Reconfigurable Real-Time systems

(Although this research area is described separately, it could evolve to deliver into the Dynamic Scene Understanding and Embedded Systems engineering activities.) Reconfigurable logic systems enable hardware components to be embedded into a system after it is deployed. This can be integrated into a design by composition framework to create a run-time co-design environment. By defining hardware components in a similar modelling framework as software components, a codesign methodology can compose systems at run time by assembling hardware and software components to meet non-functional real-time requirements (like memory footprint, power budget, real time constraints etc). Like an embedded operating system, some kind of run-time kernel could schedule both hardware and software tasks, each made up of components. For example, consider a smart camera incorporated into a visual sensor network. The camera could have it’s ‘smarts’ or embedded algorithms updated post-deployment – even real-time algorithms implemented in hardware components. This reconfiguration could be done dynamically without interrupting the operation of the camera. Software components can be migrated into hardware components by the system, depending on the changing non-functional requirements.

NICTA Capability in these areas

1) Systems and Software Engineering

The Systems and Software Engineering group has been recognised as one of three leading groups internationally in the area empirical software engineering.  The research group has established a significant capability in research in software and systems architecture.  Early research at NICTA on architecture-based performance prediction has lead to mature technology which is now being readied for commercialisation through NICTA.  Jenny Liu also co-lead the NICTA project CAmkES, which developed and released a component-based architecture for the development of embedded systems.  Work derived from the CAmkES platform is planned to be a central part of NICTA's embedded systems strategy. The group has a significant capability in the area of model-driven development, which is an approach to raise the level of abstraction in the development of systems. We also have a capability in product line engineering, an approach to achieve significant economies of scale when developing multiple related products, which benefits from our expertise in architecturally based reuse.

The team benefits from research leadership from Ross Jeffery, Jenny Liu, Liam O'Brien, Mark Staples, and Liming Zhu.

2) Embedded Real-Time Operating Systems (ERTOS)

For over 10 years, under the leadership of Prof. Gernot Heiser, the University of New South Wales and NICTA have grown one of the world’s leading real time embedded operating systems efforts. Based on earlier research done at the University of Karlsruhe, the ERTOS group at NICTA have created a verified open-source microkernel for deployment in products requiring high performance, reliable embedded applications with transient fault containment. By limiting the trusted code base of the kernel to 10,000 lines of code, a formal verification of the kernel has been possible using methodologies and tools developed in the NICTA Formal methods group in Kensington and the Logic and Computation group in Canberra. This research produced the world’s first fully verified microkernel.

One of NICTA’s most successful research programs, the ERTOS activity has resulted in a spinout company, OK Labs, trained many students and grown an operating systems research community around the University.

Key research talent in this team include Dr. Kevin Elphinstone and Dr. Gerwin Klein are all research leaders in operating systems theory and implementation, real time systems and formal methods for microkernel verification.

3) Embedded Computer Vision – Real Time Dynamic Scene Understanding

In Computer Vision, NICTA has grown a world-class capability, with a reputation as one of the top 5 groups in the world in algorithms for Dynamic Scene Understanding. Combining this with our expertise in smart camera design and the mapping of vision algorithms to hardware, we have the ability to develop advanced visual sensor networks that can detect, track and understand objects in scenes. These networks have applications in emergency response, security and surveillance as well as in traffic monitoring and flow control, crowd management, situational awareness and even augmented reality and the bionic eye.

The research leadership of Prof. Richard Hartley, Prof. Brian Lovell and Dr. Nick Barnes have grown NICTA’s capability in this area. Dr. Jian Zhang works in traffic detection for intelligent transportation systems. Dr. Abbas Bigdeli and Dr. Lars Petersson add capability in mapping vision algorithms to hardware acceleration and smart camera design. NICTA has several Ph.D. students working in these areas.

4) Wireless Systems on a Chip

In NICTA’s research group in Melbourne, Victoria, a team led by Prof. Stan Skafidas has developed millimeter wave wireless communications expertise. Stan has created a lab of researchers who have developed the world’s first fully integrated wireless radio at 60GHz in low cost CMOS technology. The team has many publications in leading RFIC conferences and is listed in a recent ABI Research report as a contender for the WirelessHD market. Dr. Gordana Felic is an expert in on-chip antenna design. Tim Walsh and Praveen Nagadouga are expert mixed signal designers and have designed multi giga-sample per second analog to digital converters. This team has been combined with a digital baseband (DBB) IC design team from LSI Australia (who joined NICTA in Nov. 2007). Together, this team will create one of the first CMOS wireless systems on a chip solution for 60GHz communications. Charles Thomas and Dr. Mark Bickerstaff are leading researchers in the DBB team, having published several large wireless system on chip ASICs in leading IEEE venues. Also in this team, Dr. Jeff Li, Dr. Adriel Kind and Dr. Nghia Ngo are world-class experts in wireless signal processing.

5) Co-Design & Reconfigurable Systems (Member and Partner Universities)

NICTA’s member and partner Universities (University of New South Wales and University of Queensland) have substantial expertise in the co-design of embedded systems, and dynamic reconfigurable hardware architectures. This expertise can be combined with NICTA’s computer vision and hardware capability to create dynamically reconfigurable secure real time sub-systems. Prof Neil Bergmann is Professor of Embedded Systems at University of Queensland (UQ) where he is a leading researcher into reconfigurable logic systems using FPGAs. Prof Sri Parameswaran is Professor of Computer Science at University of New South Wales (UNSW). Dr. Oliver Diessel (also of UNSW) is a leading researcher in dynamically reconfigurable embedded systems.