The purpose of this demonstration is to realize a proof-of-concept of the RoQME approach in a real-world example. It is a test-bed for current developments and provides valuable foundation for further developments.

The demonstration focuses on the run-time aspects of RoQME as you will see RoQME in action. The design-time part of RoQME and how RoQME will be presented to the user through tooling will be covered in an update to this demonstration (see roadmap).

The video below illustrates a typical intralogistics scenario. In this scenario the robot receives boxes from given stations and delivers them to different stations. The setting is equipped with a sensor detecting when a person enters or exits the robot working area. The video illustrates how the RoQME approach can provide metrics on non-functional properties such as safety or performance.

Disclaimer: This demonstration is a proof-of-concept of the technical feasibility. Please look at the roadmap to see how it is planned to advance this demonstration and to make it generally accessible via tooling.

The RoQME project aims to provide a comprehensive set of model-based tools enabling:

1. The specification of global robot Quality of Service (QoS) metrics defined on Non-Functional Properties (NFP). These metrics are defined in terms of the (internal and external) contextual information available to the robot; and
2. The generation (from the previous specification) of an actual RobMoSys-compliant component providing information to other components (either continuously or on demand) about the runtime evolution of the metrics previously specified.

It is worth noting that RoQME enables modeling all kinds of system-level (rather than component-level) NFP, such as, safety, performance, reliability, resource consumption, user comfort, etc. It is also worth mentioning that RoQME does not prescribe how the computed metrics are used by other components, probably in a distributed manner, as the intrinsic cross-cutting nature of NFP may scatter and have impact both in the robot behaviour and architecture.

• A new QoS Engineer role (see section Technical Details), associated with a RoQME view, specifically dedicated to the modeling of QoS metrics defined on NFPs. The RoQME View builds on:
  • The RoQME meta-model, which provides QoS Engineers with the modeling concepts needed to specify:
    • What are the relevant NFPs in a particular robotic application;
    • Which contextual information is available to the robot; and
    • How the different contexts impact the selected NFPs. These impacts will then be used to compute the QoS metrics defined on the different NFP.
  • A RoQME-to-RobMoSys mapping meta-model enabling the seamless integration of the RoQME models into the RobMoSys Ecosystem.
  • RoQME plugins for the SmartMDSD Toolchain providing:
  • A textual model editor enabling the creation and validation of QoS models by Domain Experts and QoS Engineers.
  • A model-to-code transformation enabling the generation of a metrics provider component that conforms-to the RobMoSys structures. This tool is intended to be used by QoS Engineers.
  • A QoS Metrics Service Definition enabling the use of the RoQME QoS Metrics Provider by any component in a RobMoSys conformant architecture, willing to use the computed metrics.

In the context of the RobMoSys technical user stories, this demonstration shows:

• Composable components
  • The RoQME QoS Metric provider is a composable software component generated from a RoQME model, i.e., it can be seamlessly composed into a RobMoSys-compliant system
  • The scenario was composed out of previously developed software components (e.g., components for hardware access, mapping, path planning, collision avoidance, machine-to-machine communication, box pickup/drop-off) and the newly generated RoQME QoS Metric provider component
• Traceable properties
  • RoQME supports traceable NFP, i.e., it is possible to trace which contexts are being responsible for the evolution of the metrics defined on the selected NFP. In other words, it is possible to trace back what causes the improvement or degradation of each NFP.
• Ease of use
  • RoQME provides an easy way to model QoS metrics defined on NFP
  • RoQME supports separation of roles and views (e.g. QoS Engineer focusing on NFP vs. System Architect focusing on functional aspects)
• Re-usable models
  • RoQME supports the modeling of reusable domain- and application-specific metrics on NFP
• Reliable QoS
  • RoQME enables the use of QoS metrics on NFP (e.g., for adaptation, benchmarking, etc.)
• Standardization
  • RoQME domain-models defined at tier-2 can be (re-)used in a variety of concrete tier-3 applications (e.g., standard metrics on safety defined in the context of mobile robotics can be reused/refined as standard metrics in the context of intralogistics, and these, in turn, reused/refined in different particular intralogistics applications).

The following figure outlines the role, the models and the software artifacts contributed by RoQME and how they relate to those of RobMoSys. These contributions and the links to RobMoSys are briefly described next. Further details can be found at [Vicente2018a].

Domain Experts (Tier-2) identify and agree the specification of relevant domain-specific non-functional properties. The resulting Domain-Specific RoQME Models are made available in a RobMoSys repository for being reused by other roles at Tier-3.

QoS Engineers create Application-Specific RoQME Models that gather the non-functional aspects to be measured in the robotics system under development while, in parallel, the System Architect describes its functionality. The QoS Engineer may create application-specific RoQME models from scratch or by reusing and extending some of the domain-specific RoQME models already available in the RobMoSys repository. The models developed by the QoS Engineer specify (1) the non-functional properties considered relevant for the application being developed; (2) the contextual information available to the robot; and (3) a number of observations describing how (2) impacts (1). These observations will enable the estimation of the QoS metrics defined on the selected non-functional properties.

The QoS Engineer uses the RoQME Generator to create a QoS Metrics Provider component from the previous model. The resulting component is made available to the System Builder, who integrates it into the RobMoSys application architecture by (1) connecting it to the corresponding context providers, and (2) to any other component willing to use the computed metrics (if any).

The figure included below shows a screenshot of the system component architecture diagram, as modeled in the SmartMDSD Toolchain. The system was composed from previously developed software components.

The following picture shows the QoS Metrics Provider component used in the demonstration. The hull of this component was developed using the SmartMDSD Toolchain, while its internals were manually implemented using the support software libraries developed in RoQME for context monitoring, complex event processing, and metrics estimation. Further developments in RoQME will realize the full generation of this component based on the RoQME models (see roadmap). As illustrated in the picture, the component receives contexts from other components in the system (context providers) and processes them according to the RoQME approach (lower left). It then provides the estimated QoS metrics to other components for further processing or adaptation (e.g., by the Sequencer).

The RoQME model detailed next specifies: (1) the non-functional properties identified as relevant for the experiment and for which the RoQME runtime infrastructure will provide estimations (metrics) in the range [0, 1]; (2) the context data provided by the robot; and (3) the observations which, based on the specified context patterns, will more or less impact (reinforcing or undermining) the previous non-functional properties. This model uses a preliminary syntax, which is subject to improvement in the final textual model editor that is currently being developed in Rome.

• [SAFETY]: degree to which the robot is successful in producing the following desired results:
  • The robot does not bump
  • The velocity of the robot is bounded (lower than V ) when there is a person in the scenario area.

It is worth mentioning that the reference value for SAFETY is 1, that is, a priori, the system is considered safe and, only if there is evidence against this belief, the value of the safety metric is reduced accordingly. The reference value of a property is its a-priori probability of being optimal, and is the value the property tends to when there is no evidence making it improve (reinforcing evidence) or worsen (undermining evidence). Note that most properties use a reference value equal to 0,5.

• [PERFORMANCE]: degree to which the robot is successful in producing the following desired results:
  • The robot completes the tasks successfully (i.e., tasks are not aborted and the robot does not enter into an ERROR state).
  • The robot completes each task within a given time-slot (MAX_JOB_TIME)

Contexts:

• Bump : Event
• Velocity: number (unit: m/s)
• Robot_State : Enum {
  IDLE_NOT_CHARGING,
  IDLE_CHARGING,
  BUSY_DRIVING_WITH_LOAD,
  BUSY_DRIVING_EMPTY,
  ERROR
  }
• Job_State : Enum {
  NOT_STARTED,
  STARTED,
  FINISHED,
  ABORTED
  }

Derived context (from Job_State):

• time_jobDone: Number > 0 (unit: s)

This derived context is calculated as the time passed since a job starts (Job_State::STARTED) until it finishes (Job_State::FINISHED).

• Person_State: Enum {
  IN,
  OUT
  }

• O1. Bump undermines [SAFETY] VERY_HIGH
• O2. Velocity > V && Person_State::PERSON_IN undermines [SAFETY] VERY_HIGH
• O3. Job_State:: JOB_FINISHED && time_jobDone < MAX_JOB_TIME =⇒ reinforces [PERFORMANCE] HIGH
• O4. Robot_State::ERROR undermines [PERFORMANCE]
• O5. Job_State::ABORTED undermines [PERFORMANCE]

The story starts with the robot in an IDLE state and placed in its initial position. There is an operator ready to handover boxes to the robot.

1. The robot is ordered a new task (Job_State::STARTED) so it moves to the handover position (Robot_State::BUSY_DRIVING_EMPTY). When it is ready, the operator places a box on top of it. The robot moves to the delivery position (Robot_State::BUSY_DRIVING_WITH_LOAD) and puts the box on the belt. The task is completed successfully (Job_State::FINISHED) and within the required time-slot ⇒ Observation O3 is fired and PERFORMANCE improves.
2. A visitor enters the room and meets the operator (Person_State::IN). Simultaneously, the robot is ordered a new task (Job_State::STARTED) so it starts moving again to the handover position (Robot_State::BUSY_DRIVING_EMPTY). On its way, the robot moves faster than allowed when a person is in the robot working area ⇒ Observation O2 is fired and SAFETY is significantly worsen.
3. The visitor, not being aware of the robot, bumps into it (Bump) when turning around for leaving the room ⇒ Observation O1 is fired and SAFETY is significantly worsen.
4. Immediately after the bumping, the robot enters into the ERROR state (Robot_State::ERROR) ⇒ Observation O4 is fired and PERFORMANCE is worsen.
5. After the error, the task is aborted (Job_State::ABORTED) ⇒ Observation O5 is fired and PERFORMANCE is worsen.
6. The robot is ordered a new task (Job_State::STARTED) which is successfully completed (Job_State::FINISHED) ⇒ Observation O3 is fired and PERFORMANCE improves.

RoQME provides a visualization tool that displays at run-time: (1) the contextual information obtained from the robot; (2) the observations fired when the context patterns identified in the RoQME model are detected; and (3) the estimation of the metrics defined on the non-functional properties, as they vary according to the previous observations. The following screenshots show the graphics produced by the visualization tool during the experiment for the SAFETY (top) and the PERFORMANCE (bottom) non-functional properties, together with the observations with impact on their estimations. Labels (numbered circles) referring to the different story steps described in the previous section have been manually added to make the experiment timeline clearer.

RoQME also provides a module for recording the data generated while running real-world experiments (i.e., experiments that involve real robots). It records: a) the context data generated by the robot/s; b) the identified observations, and c) the QoS metric estimations. The recorded data can then be used for simulating the experiment as many times as needed without the need of using the real robot/s again. Besides, this information can also be used, for instance, for benchmarking purposes.

The core functionality of collecting contexts and processing metrics on non-functional properties is already operational in RoQME although, at the moment, the QoS metrics provider component is implemented manually.

The following items are currently being developed and are expected to be available and usable in the form of RoQME plugins for the SmartMDSD Toolchain. Its expected release date is March 2019. It will cover:

• Modeling support for the Domain Expert to model QoS Domain Models (Tier 2)
• Modeling support for the QoS Engineer to model QoS Models (Tier 3)
• A number of code generators for automatically deriving all the RoQME software artifacts, in particular, the hull and the internals of the QoS Metrics Provider Component. In case the complex event processor or the reasoner (or both), need to be executed out of the QoS Metrics Provider Component, i.e., out of the robot (in different computers), they will be also generated as independent artifacts.

All the RoQME partners are pledged to making the knowledge generated in the course of the Project as widely and freely available as possible for subsequent research and development. For this reason, the RoQME partners are fully committed to open-access and open-source policies.

To discuss this demonstration, see: https://discourse.robmosys.eu/t/demonstration-dealing-with-metrics-on-non-functional-properties-in-robmosys-roqme-itp

This technical demonstration has been realized by the RoQME project team and the Ulm University of Applied Sciences. The methods and tools behind the estimation of the non-functional properties are being developed by the RoQME ITP (University of Extremadura, University of Málaga and Biometric Vox). The SmartMDSD Toolchain and the intralogistics use-case/pilot application are being developed by the Ulm University of Applied Sciences.

RoQME is an Integrated Technical Project (ITP) of EU H2020 RobMoSys (robmosys.eu). Ulm University of Applied Sciences is a RobMoSys core partner.

This activity has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 732410.

Fur further information and to learn more about the concepts used here, see:

• The SmartSoft World and the The SmartMDSD Toolchain, the tool that was used to develop the software components and to compose the application.
• This scenario uses software components from the Intralogistics Industry 4.0 Robot Fleet Pilot
• RoQME Integrated Technical Project.

You can follow the RoQME Project updates at:

• The ReasearchGate RoQME Project
• The Linked RoQME Group
• The RoQME Twitter account
• The RoQME section in the RobMoSys website
• The Community Corner section in the RobMoSys wiki

[Espin2018] Espín, J. M., Font, R., Ingles-Romero, J. F. & Vicente-Chicote, C. Towards the Application of Global Quality-of-Service Metrics in Biometric Systems. IberSPEECH 2018, 21-23 November 2018, Barcelona (Spain). Download. Download

[Ingles2018a] Inglés-Romero, J. F., Spín, J. M., Jiménez, R., Font, R. & Vicente-Chicote, C. Towards the Use of Quality of Service Metrics in Reinforcement Learning: A Robotics Example. 5th International Workshop on MOdel-driven Robot Software Engineering (MORSE 2018), in conjunction with MODELS 2018, 15 October 2018, Copenhagen (Denmark). Download. Download

[Ingles2018] Inglés-Romero, J. F. How well does my robot work? RoQME: A project aimed at measuring quality of service in robotics. April 2018. Available at Medium.com. Download

[Lutz2019] Lutz, M., Inglés-Romero, J. F., Stampfer, D., Lotz, A., Vicente-Chicote, C., & Schlegel, C. (2019). Managing Variability as a Means to Promote Composability: A Robotics Perspective. In A. Rosado da Cruz, & M. Ferreira da Cruz (Eds.), New Perspectives on Information Systems Modeling and Design (pp. 274-295). Hershey, PA: IGI Global. DOI: 10.4018/978-1-5225-7271-8.ch012 Download

[Vicente2018a] Vicente-Chicote, C., Inglés-Romero, J. F., Martínez, J., Stampfer, D., Lotz, A., Lutz, M. & Schlegel, C. A Component-Based and Model-Driven Approach to Deal with Non-Functional Properties through Global QoS Metrics. 5th International Workshop on Interplay of Model-Driven and Component-Based Software Engineering (ModComp 2018), in conjunction with MODELS 2018, 14 October 2018, Copenhagen (Denmark). . Download

[Vicente2018b] Vicente-Chicote, C., Berrocal, J., García-Alonso, J. M., Hernández, J., Bandera, A., Martínez, J., Romero-Garcés, A., Font, R. & Inglés-Romero, J. F., (2018). RoQME: Dealing with Non-Functional Properties through Global Robot QoS Metrics. SISTEDES Conference, 17-19 September 2018, Sevilla (Spain). Download