How Do I Know I Have Timing Issues?

Timing problems are not found; timing problems find you. In automotive applications, especially those of high complexity, they actually seem to appear as issues with the system’s functionality. System integrators describe them in stories about lost CAN data, or data that seems to be received twice. Others relate how seemingly random failures occur on some ECUs, but always after a set period of time of several minutes.

Over the years, INCHRON has supported its automotive customers in resolving such issues, and none of them has been to shown to be due to a random failure. Instead, each can be perfectly defined and related back to some timing related issue, allowing it to be reproduced. Lack of synchronization in the time base between ECUs, poor scheduling choices in the operating system, or too high a processor load during a critical moment on a core of a multi-core processor: all these reasons and more result in what look like functionality issues but, in reality, are timing issues.

Technical Indicators for Timing Issues

Timing issues are exceptional hard to find because they disguise themselves as functionality issues. Here we review some problems seen in system functionality that turned out to be timing issues:

Sensor fusion in ADAS application seems to work, but its output occasionally seems to be incorrect – this occurs when some of the data sampled is allowed to ‘go stale’, i.e. it has aged compared to the data with which it is fused. Typically, data from one of the sensors is not being processed in a timely manner, often due to heavy processor load or the processing of interrupts.
Actuators are not always activated in the timeframe expected – once a decision is made on one ECU to activate the brakes, attached over a network to another ECU, they need to be applied within a specific amount of time if safety is to be ensured. Should any ECU along the chain be suffering from timing-related issues, this could, on occasion, delay their application.
Interrupts seem to be being dropped – external events are functioning correctly, but the related interrupt routines do not seem to be activated every time as expected. The priorities of the interrupts, or their suspension in runnables or tasks, could be the cause here, starving the interrupt of the time required to acquire and process each event as it happens.

Signs That the Development Process Has Not Fully Considered Timing Implications

Automotive project teams can quickly determine the requirements for their application from a functional perspective, but often have trouble when it comes to defining timing requirements. The opinion is often that timing requirements cannot be defined before the software is created. If timing requirements appear at all, they are often limited to the most simple of statements. Functional issues that later turn out to be timing issues will be found during software and system integration when:

Processor performance is considered to be ‘more than enough’ from the outset – the reality is, it never is. Software grows to fill the available resources. Trying to work timing requirements back into a project that started without them is an almost impossible task. It impacts every decision made. Assume that timing issues will happen, then plan how best to tackle them through every stage of the project.
Model-based design approach shows little understanding for dynamic system behavior – models are only as good as the sum of their parts. If is assumes synchronicity between sensors and ECU, between ECUs, and ECU and actuators, the model will never fail. Considering the realities of unsynchronized signals and differing processor clock speeds is essential to the establishment of a reliable model.
Results of timing-related testing are not included on Continuous Integration (CI) dashboards – if timing requirements are being tested, they need to show as highly visible pass/fail results on CI dashboards. Only in this way can the issues caused by timing problems be found and resolved before the project progresses too far.

If you would like to find out more, why not get in touch with our team to discover how we can help?