Skip to content

Terms and their meanings

Common Terms

xRally is testing framework and tool that unifies all testing approaches: functional, concurrency, regression, performance, load, scale and even chaos using composition of plugins. It's designed to solve problem of testing complex platforms and it's suitable for testing of any platforms or service: like OpenStack, Kubernetes, Mesos, Docker, AWS, Google Cloud, Azure, (put your here)

Term Meaning
Alembic Lightweight database migration tool which powers Rally migrations. Read more at Official Alembic documentation
DB Migrations Rally supports database schema and data transformations, which are also known as migrations. This allows you easily update to newer version of Rally keeping all your data in sync.
xRally config Rally behavior can be customized by editing its configuration file, rally.conf*, in configparser format . While being installed, Rally generates a config with default values from its sample When started, Rally searches for its config in next directories: "/etc/rally/rally.conf", "~/.rally/rally.conf", "/etc/rally/rally.conf"
Rally DB Rally uses a relational database as data storage. Several database backends are supported: SQLite (default), PostgreSQL, and MySQL. The database connection can be set via the configuration file option [database]/connection
Rally Plugin Rally is heavily plugable product. It's core is relative small, that just glue and executes properly plugins and persist the results to DB. Plugins are responsible to do actually work, like calling APIs, checking results, exporting data and so on. Plugins are just regular python classes grouped by their type (base plugin) which defines interface that should be implemented. Read more about plugins here

Env Terms

Env component is one of key component in xRally, It manages and stores information about tested platforms. Env manager is using platform plugins to: create, delete, cleanup, check health, obtain information about platforms. Every Env consists of:

  • unique name and UUID
  • dates when it was created and updated
  • default config override
  • platform plugins spec which are used to create platform
  • as well as platform & plugin data which are used by other platform commands

Env data is consumed by other rally components, like task and verify

Term Meaning
Env Spec YAML file with with complete information about env, used by env create command to create env record
Env Platform Plugin that is used to manage platform data (creds) and state. Samples of platforms: OpenStack, Kubernetes, Docker, AWS, Azure, Libvirt, ...

Task Terms

Task component is responsible for executing tests defined in task specs, persisting and reporting results.

Term Meaning
Task Plugin: Context Context plugins are responsible for creating environment in which scenario is run. For example, creating users, adding roles, setting up quotas and so on
Task Plugin: Runner Plugin tha executes scenario plugin multiple time to achieve required load profile.
Task Plugin: Scneario Set of actions by user against the platform. Authenticate, Create product, delete product. Scenario is executed by runner multiple time inside the context created by contexts plugins.
Task Plugin: SLA Plugin that recieves the results of scenario execution and decides whatever results are matching SLA
Task Plugin: Hook Plugin that allows to introduce custom action after some iteration or some time. Makes it possible to do chaos and performance testing together!
Task Iteration One of scenario runs, sometimes points order number of scenario execution used by task runner.
Task Spec A file that describes how to run a Rally Task. It can be in JSON or YAML format. The rally task start command needs this file to run the task. The input task is pre-processed by the Jinja2 templating engine so it is very easy to create repeated parts or calculate specific values at runtime. It is also possible to pass values via CLI arguments, using the --task-args or --task-args-file options.
Task Spec: Subtask Task consists of multiple subtasks, basically subtask is just a test case
Task Spec: Workload Every subtask consists of workloads, and how to execute workloads (serial or parallel). Workload combines scenario, runner, context and SLA defining minimal executable entity.

Verify Terms

Verify Component allows to wrap subunit-based testing tools and provide complete tool on top of them with allow to do pre configuration, post cleanup as well process and persist results to Rally DB for future use like reporting and results comparing.

Term Meaning
Verification Process of execution xrally verify command that runs underlying tool, collecting results and storing them in xRally database