SIMICS OS Awareness Tracker – Go Fetch!

In our previous article we mentioned that a tracker extracts information from the OS to identify the various data structures. In the case of FreeRTOS, this consists of linked lists and associated TCB.

The tracker implements a function to extract symbols from the executable (like a debugger) to locate the data. As FreeRTOS is a simple environment and does not use MMUs and per-process data structures, parsing the executable is all we need to do to get the symbols.

In more complex OSes like Linux, extracting symbols is a starting point, but is not enough. You need to boot the OS and parse structures based on both symbols and memory locations derived from some live structures. The live analysis would be performed by a detector. We are not going to cover implementing a detector at this point.

Extracting the OS data is implemented with the detect-parameters command. Like all commands for the simics CLI, it is a python function. It is in the module_load.py file, which executes automatically when the tracker module is loaded by Simics.

The data extracted is stored in a python dictionary. The key is a string representing the item name and the data is typically a string (like the version of the OS) or a number (like the pointer to a data structure or an offset of some kind).

Once we have the data, it is saved to a parameter file (a JSON-like representation of the dictionary, using the python built in structure save). When executing the launch script, the parameter file is loaded with the load-parameters command. This reads the parameter file and loads the information in the tracker.

The reason for the two step process (detect, then load) is that the parameter file content does not typically change between runs. The data structures are always at the same location in memory. Unless the OS executable is modified, the parameters simply don’t change. It is much simpler to read values from a file than to scour the symbol file and navigate data structures in memory.

Here is a walkthrough of our simplified implementation of the FreeRTOS parameter detection function:

def get_freertos_params(self, symbol_file):
  tcf_elf = get_tcf_agent(tcfc).iface.tcf_elf
  (ok, elf_id) = tcf_elf.open_symbol_file(symbol_file, 0)
  if not ok:
    raise CliError(elf_id)

Remember that we need to extract the address of some global data structures from the executable file. In fact, for FreeRTOS, that is all we need to do. It turns out that Simics’ TCF framework contains a handy dandy way to parse symbols files.
This is exactly what these first few lines do. This is the equivalent of going to the Simics Eclipse interface and adding a symbol file in the symbol Explorer view.

params = {}
params['rtosVersion'] = "FreeRTOS 10"
params['sys_prio'] = 5     # Number of system Priorities

params is the dictionary we will save to a file. These first few entries represent elements that are statically configured and won’t change, or elements that we can’t discover dynamically. In this case, the version of FreeRTOS (a #define never stored in a variable) and the number of task priorities.

symbols = ['pxReadyTasksLists', 'xDelayedTaskList1', 'pxCurrentTCB']

for (param in symbols):
    (ok, info) = tcf_elf.get_symbol_info(elf_id, param)
    if not ok:
        raise CliError("%s not found in '%s'" % (symbol, symbol_file))
    params[param] = info[0]

This section extracts the symbol values from the elf file. We iterate over the list of symbol we want to extract and use the tcf framework’s get_symbol_info interface to extract the symbol offset (info[0]) and store it in the dictionary.

fields = {'task_name': ('tskTCB', 'pcTaskName'),
          'task_prio': ('tskTCB', 'uxPriority'),
          'tcb_offset': ('ListItem_t', 'pvOwner') 
         }
for (param, (symbol, field)) in list(fields.items()):
    (ok, info) = tcf_elf.get_field_offset(elf_id, symbol, field)
    if not ok:
        raise CliError("%s.%s not found in '%s'" % (symbol,field, symbol_file))
    else:
        params[param] = info[0]

It is one thing to know where a data structure is in memory, but we also need to know where in this structure the information we want is located. The get_field_offset gets us what we need. It takes a struct name and field, and returns the field offset (and size). It is then saved in the params dictionary.

We could analyze the code and hardcode the offset either in the tracker itself or as a static dictionary entry (like the sys_prio above) but this creates a brittle tracker. What if the OS has configuration options available that change the layout of the structure? With get_field_offset, we an insulated from such changes.

tcf_elf.close_symbol_file(elf_id)
return ['freertos_tracker', params]

Once we are done building the parameter list, we simply return it along with the tracker identifier string. The string is used by the load-parameter to validate that we are reading the parameters for the correct tracker.

Here is the detect_parameters function:

def detect_parameters_cmd(self, symbol_file, param_file):
  params = get_freertos_params(self, symbol_file)
  with open(param_file, 'w') as f:
    f.write('# -*- Python -*-\n'
            + repr(params)
            + '\n')

A simplified load-parameter function is similar to the following:

def load_parameters(self, filename):
  with open(filename, 'r') as f:
    s = f.read()

  if s.startswith('# -- Python --'):
    params = ast.literal_eval(s)
  else:
    raise CliError('Unknown file format')

  if self.classname == "freertos_tracker_comp":
        self.iface.osa_parameters.set_parameters(params)
  else:
    if (params[0] != "freertos_tracker"):
        raise CliError("Unsupported parameters type '%s'" % params[0])
    self.params = params[1]
  return

Notice that load_parameters works with both the OS Awareness component and the tracker. Hence the check on the classname to determine which object command was invoked. software.tracker.load-parameters or software.tracker.tracker.load-parameters.

In the next installment, we will take a closer look at the structure of the tracker and explore the difference between the tacker and the tracker_comp.

SIMICS OS Awareness Tracker – Intro

In this series of posts, we will explore the various aspects of implementing a OS tracker for Simics OS Awareness (OSA) component. The goal of a tracker is to allow Simics to keep track of the state of an OS. This includes knowing what tasks are present in the system, provide a context to allow source debugging of a task, know which task is executing on which CPU, track the CPU virtual memory to physical memory mapping and more.

Not only is a tracker specific to a given OS, but it must also be aware of the CPU architecture it runs on. This is especially important when the OS makes use of Virtual Memory via MMU. The Page table organization on an ARM processor is very different than an X86. In fact, the X86 could have multiple more (Linear address vs. segmented, etc…). Depending on the OS, the tracker might need to be aware of all those CPU variations. In addition, the tracker might need to keep track of whether the CPU is in Supervisory mode (typical for Kernel operations) or in User Mode (your typical executable program).

With a complex OS like Linux, the tracker might need to keep track of multiple levels of memory management.

From a Simics user perspective, the OS Awareness is represented as a Tree, where each end node is a different thread or program. Intermediate nodes represent different modes / layers of the OS (Kernel vs. User space for example).
Here is an example of a Linux tree:

SIMICS OS Awareness

In this system, there are a number of kernel threads and user space programs running.

In order for the OSA Tracker to represent the state of the OS, it must be able to analyze the OS scheduler’s data structures. By extension, it implies that whoever implements a tracker must understand those data structures and how they are used.
Not every elements of the scheduler needs to be processed by the tracker, only those related with identifying the various contexts and the state of those tasks.

Generally speaking, the tracker cares about:

  • List of threads in the kernel
  • List of Memory Contexts / User Programs
  • Within a User Program, list of threads
  • State of each individual threads/programs
  • For each running thread, which CPU it is executing on

To keep things simple, we will implement a tracker for FreeRTOS, a MIT licensed embedded operating system for small microcontrollers. This will be a single core, MMUless configuration. Thus, we will avoid having to deal with processor specific MMU analysis.

As we described above, before we can start implementing the tracker, we need to understand the data structures we need to track.

In FreeRTOS, the Task Control Block has the following elements the tracker cares about:

typedef struct tskTaskControlBlock
{
…
UBaseType_t uxPriority; /*< The priority of the task. 0 is the lowest priority. */
…
char pcTaskName[ configMAX_TASK_NAME_LEN ]; /*< Descriptive name given to the task when created. Facilitates debugging only. */
…
} tskTCB;

For our simple FreeRTOS tracker, we will only consider the list of active tasks for each priority levels and the list of tasks that are delayed.

PRIVILEGED_DATA static List_t pxReadyTasksLists[ configMAX_PRIORITIES ]; /*< Prioritised ready tasks. */
PRIVILEGED_DATA static List_t xDelayedTaskList1; /*< Delayed tasks. */

Here are the relevant definitions for the list structures:

typedef struct xLIST
{
…
volatile UBaseType_t uxNumberOfItems;
ListItem_t * pxIndex; /*< Used to walk through the list. */
MiniListItem_t xListEnd; /*< List item that is always at the end of the list used as a marker. */
…
} List_t;

typedef struct xLIST_ITEM
{
…
struct xLIST_ITEM * pxNext; /*< Pointer to the next ListItem_t in the list. */
struct xLIST_ITEM * pxPrevious; /*< Pointer to the previous ListItem_t in the list. */
void * pvOwner; /*< Pointer to the object (normally a TCB) that contains the list item. */
…
} ListItem_t;

The tracker will need to parse the lists and extract the task names and priorities to discover the tasks present in the system and build the Tree representation.

In addition, the tracker also needs to monitor the list structures to detect if a new task has been created or an existing task has been deleted. This allows the tracker to maintain an accurate Tree even in a dynamic task environment.

In the next installment, we will start the implementation of our tracker.