Skip to content

In-tree plugins.

Processed releases: rally 0.11.0 - 1.2.1

Task Component

Chart

Base class for charts.

This is a base for all plugins that prepare data for specific charts in HTML report. Each chart must at least declare chart widget and prepare data that is suitable for rendering by JavaScript.

Lines [Chart]

Display results as generic chart with lines.

This plugin processes additive data and displays it in HTML report as linear chart with X axis bound to iteration number. Complete output data is displayed as linear chart as well, without any processing.

Examples of using this plugin in Scenario, for saving output data:

self.add_output(
    additive={"title": "Additive data as stacked area",
              "description": "Iterations trend for foo and bar",
              "chart_plugin": "Lines",
              "data": [["foo", 12], ["bar", 34]]},
    complete={"title": "Complete data as stacked area",
              "description": "Data is shown as stacked area, as-is",
              "chart_plugin": "Lines",
              "data": [["foo", [[0, 5], [1, 42], [2, 15], [3, 7]]],
                       ["bar", [[0, 2], [1, 1.3], [2, 5], [3, 9]]]],
              "label": "Y-axis label text",
              "axis_label": "X-axis label text"})

Platform: default

Module: rally.task.processing.charts


Pie [Chart]

Display results as pie, calculate average values for additive data.

This plugin processes additive data and calculate average values. Both additive and complete data are displayed in HTML report as pie chart.

Examples of using this plugin in Scenario, for saving output data:

self.add_output(
    additive={"title": "Additive output",
              "description": ("Pie with average data "
                              "from all iterations values"),
              "chart_plugin": "Pie",
              "data": [["foo", 12], ["bar", 34], ["spam", 56]]},
    complete={"title": "Complete output",
              "description": "Displayed as a pie, as-is",
              "chart_plugin": "Pie",
              "data": [["foo", 12], ["bar", 34], ["spam", 56]]})

Platform: default

Module: rally.task.processing.charts


StackedArea [Chart]

Display results as stacked area.

This plugin processes additive data and displays it in HTML report as stacked area with X axis bound to iteration number. Complete output data is displayed as stacked area as well, without any processing.

Keys "description", "label" and "axis_label" are optional.

Examples of using this plugin in Scenario, for saving output data:

self.add_output(
    additive={"title": "Additive data as stacked area",
              "description": "Iterations trend for foo and bar",
              "chart_plugin": "StackedArea",
              "data": [["foo", 12], ["bar", 34]]},
    complete={"title": "Complete data as stacked area",
              "description": "Data is shown as stacked area, as-is",
              "chart_plugin": "StackedArea",
              "data": [["foo", [[0, 5], [1, 42], [2, 15], [3, 7]]],
                       ["bar", [[0, 2], [1, 1.3], [2, 5], [3, 9]]]],
              "label": "Y-axis label text",
              "axis_label": "X-axis label text"})

Platform: default

Module: rally.task.processing.charts


StatsTable [Chart]

Calculate statistics for additive data and display it as table.

This plugin processes additive data and compose statistics that is displayed as table in HTML report.

Examples of using this plugin in Scenario, for saving output data:

self.add_output(
    additive={"title": "Statistics",
              "description": ("Table with statistics generated "
                              "from all iterations values"),
              "chart_plugin": "StatsTable",
              "data": [["foo stat", 12], ["bar", 34], ["spam", 56]]})

Platform: default

Module: rally.task.processing.charts


Table [Chart]

Display complete output as table, can not be used for additive data.

Use this plugin for complete output data to display it in HTML report as table. This plugin can not be used for additive data because it does not contain any processing logic.

Examples of using this plugin in Scenario, for saving output data:

self.add_output(
    complete={"title": "Arbitrary Table",
              "description": "Just show columns and rows as-is",
              "chart_plugin": "Table",
              "data": {"cols": ["foo", "bar", "spam"],
                       "rows": [["a row", 1, 2], ["b row", 3, 4],
                                ["c row", 5, 6]]}})

Platform: default

Module: rally.task.processing.charts


TextArea [Chart]

Arbitrary text.

This plugin processes complete data and displays of output in HTML report.

Examples of using this plugin in Scenario, for saving output data:

self.add_output(
    complete={"title": "Script Inline",
              "chart_plugin": "TextArea",
              "data": ["first output", "second output",
                       "third output"]]})

Platform: default

Module: rally.task.processing.charts


Context

dummy_context [Context]

Dummy context.

Platform: default

The input of this plugin should be valid to the following JSONSchema
{
    "$schema": "http://json-schema.org/draft-04/schema", 
    "type": "object", 
    "properties": {
        "fail_cleanup": {
            "type": "boolean"
        }, 
        "fail_setup": {
            "type": "boolean"
        }
    }, 
    "additionalProperties": false
}

Module: rally.plugins.common.contexts.dummy


Hook Action

Factory for hook classes.

sys_call [Hook Action]

Performs system call.

Platform: default

The input of this plugin should be valid to the following JSONSchema
{
    "$schema": "http://json-schema.org/draft-04/schema", 
    "type": "string", 
    "description": "Command to execute."
}

Module: rally.plugins.common.hook.sys_call


Hook Trigger

Factory for hook trigger classes.

event [Hook Trigger]

Triggers hook on specified event and list of values.

Platform: default

The input of this plugin should be valid to the following JSONSchema
{
    "$schema": "http://json-schema.org/draft-04/schema", 
    "type": "object", 
    "oneOf": [
        {
            "description": "Triage hook based on specified seconds after start of workload.", 
            "properties": {
                "at": {
                    "type": "array", 
                    "items": {
                        "type": "integer", 
                        "minimum": 0
                    }, 
                    "minItems": 1, 
                    "uniqueItems": true
                }, 
                "unit": {
                    "enum": [
                        "time"
                    ]
                }
            }, 
            "additionalProperties": false, 
            "required": [
                "unit", 
                "at"
            ]
        }, 
        {
            "description": "Triage hook based on specific iterations.", 
            "properties": {
                "at": {
                    "type": "array", 
                    "items": {
                        "type": "integer", 
                        "minimum": 1
                    }, 
                    "minItems": 1, 
                    "uniqueItems": true
                }, 
                "unit": {
                    "enum": [
                        "iteration"
                    ]
                }
            }, 
            "additionalProperties": false, 
            "required": [
                "unit", 
                "at"
            ]
        }
    ]
}

Module: rally.plugins.common.hook.triggers.event


periodic [Hook Trigger]

Periodically triggers hook with specified range and step.

Platform: default

The input of this plugin should be valid to the following JSONSchema
{
    "$schema": "http://json-schema.org/draft-04/schema", 
    "type": "object", 
    "oneOf": [
        {
            "description": "Periodically triage hook based on elapsed time after start of workload.", 
            "properties": {
                "end": {
                    "type": "integer", 
                    "minimum": 1
                }, 
                "start": {
                    "type": "integer", 
                    "minimum": 0
                }, 
                "step": {
                    "type": "integer", 
                    "minimum": 1
                }, 
                "unit": {
                    "enum": [
                        "time"
                    ]
                }
            }, 
            "additionalProperties": false, 
            "required": [
                "unit", 
                "step"
            ]
        }, 
        {
            "description": "Periodically triage hook based on iterations.", 
            "properties": {
                "end": {
                    "type": "integer", 
                    "minimum": 1
                }, 
                "start": {
                    "type": "integer", 
                    "minimum": 1
                }, 
                "step": {
                    "type": "integer", 
                    "minimum": 1
                }, 
                "unit": {
                    "enum": [
                        "iteration"
                    ]
                }
            }, 
            "additionalProperties": false, 
            "required": [
                "unit", 
                "step"
            ]
        }
    ]
}

Module: rally.plugins.common.hook.triggers.periodic


SLA

Factory for criteria classes.

failure_rate [SLA]

Failure rate minimum and maximum in percents.

Platform: default

The input of this plugin should be valid to the following JSONSchema
{
    "$schema": "http://json-schema.org/draft-04/schema", 
    "type": "object", 
    "properties": {
        "max": {
            "type": "number", 
            "minimum": 0.0, 
            "maximum": 100.0
        }, 
        "min": {
            "type": "number", 
            "minimum": 0.0, 
            "maximum": 100.0
        }
    }, 
    "minProperties": 1, 
    "additionalProperties": false
}

Module: rally.plugins.common.sla.failure_rate


max_avg_duration [SLA]

Maximum average duration of one iteration in seconds.

Platform: default

The input of this plugin should be valid to the following JSONSchema
{
    "type": "number", 
    "minimum": 0.0, 
    "exclusiveMinimum": true
}

Module: rally.plugins.common.sla.max_average_duration


max_avg_duration_per_atomic [SLA]

Maximum average duration of one iterations atomic actions in seconds.

Platform: default

The input of this plugin should be valid to the following JSONSchema
{
    "$schema": "http://json-schema.org/draft-04/schema", 
    "type": "object", 
    "patternProperties": {
        ".*": {
            "type": "number", 
            "description": "The name of atomic action."
        }
    }, 
    "minProperties": 1, 
    "additionalProperties": false
}

Module: rally.plugins.common.sla.max_average_duration_per_atomic


max_seconds_per_iteration [SLA]

Maximum time for one iteration in seconds.

Platform: default

The input of this plugin should be valid to the following JSONSchema
{
    "type": "number", 
    "minimum": 0.0, 
    "exclusiveMinimum": true
}

Module: rally.plugins.common.sla.iteration_time


outliers [SLA]

Limit the number of outliers (iterations that take too much time).

The outliers are detected automatically using the computation of the mean and standard deviation (std) of the data.

Platform: default

The input of this plugin should be valid to the following JSONSchema
{
    "$schema": "http://json-schema.org/draft-04/schema", 
    "type": "object", 
    "properties": {
        "max": {
            "type": "integer", 
            "minimum": 0
        }, 
        "min_iterations": {
            "type": "integer", 
            "minimum": 3
        }, 
        "sigmas": {
            "type": "number", 
            "minimum": 0.0, 
            "exclusiveMinimum": true
        }
    }, 
    "additionalProperties": false
}

Module: rally.plugins.common.sla.outliers


performance_degradation [SLA]

Calculates performance degradation based on iteration time.

This SLA plugin finds minimum and maximum duration of iterations completed without errors during Rally task execution. Assuming that minimum duration is 100%, it calculates performance degradation against maximum duration.

Platform: default

The input of this plugin should be valid to the following JSONSchema
{
    "$schema": "http://json-schema.org/draft-04/schema", 
    "type": "object", 
    "properties": {
        "max_degradation": {
            "type": "number", 
            "minimum": 0.0
        }
    }, 
    "additionalProperties": false, 
    "required": [
        "max_degradation"
    ]
}

Module: rally.plugins.common.sla.performance_degradation


Scenario

This is base class for any scenario.

All Scenario Plugins should be subclass of this class.

Dummy.dummy [Scenario]

Do nothing and sleep for the given number of seconds (0 by default).

Dummy.dummy can be used for testing performance of different ScenarioRunners and of the ability of rally to store a large amount of results.

Platform: default

Parameters
Argument Description
sleep [ref] idle time of method (in seconds).

Module: rally.plugins.common.scenarios.dummy.dummy


Dummy.dummy_exception [Scenario]

Throws an exception.

Dummy.dummy_exception used for testing if exceptions are processed properly by task engine and analyze rally results storing & displaying capabilities.

Platform: default

Parameters
Argument Description
size_of_message [ref] int size of the exception message
sleep [ref] idle time of method (in seconds).
message [ref] message of the exception

Module: rally.plugins.common.scenarios.dummy.dummy


Dummy.dummy_exception_probability [Scenario]

Throws an exception with given probability.

Dummy.dummy_exception_probability used for testing if exceptions are processed properly by task engine and analyze rally results storing & displaying capabilities.

Platform: default

Parameters
Argument Description
exception_probability [ref] Sets how likely it is that an exception will be thrown. Float between 0 and 1 0=never 1=always.

Module: rally.plugins.common.scenarios.dummy.dummy


Dummy.dummy_output [Scenario]

Generate dummy output.

This scenario generates example of output data.

Platform: default

Parameters
Argument Description
random_range [ref] max int limit for generated random values

Module: rally.plugins.common.scenarios.dummy.dummy


Dummy.dummy_random_action [Scenario]

Sleep random time in dummy actions.

Platform: default

Parameters
Argument Description
actions_num [ref] int number of actions to generate
sleep_min [ref] minimal time to sleep, numeric seconds
sleep_max [ref] maximum time to sleep, numeric seconds

Module: rally.plugins.common.scenarios.dummy.dummy


Dummy.dummy_random_fail_in_atomic [Scenario]

Dummy.dummy_random_fail_in_atomic in dummy actions.

Can be used to test atomic actions failures processing.

Platform: default

Parameters
Argument Description
exception_probability [ref] Probability with which atomic actions fail in this dummy scenario (0 <= p <= 1)

Module: rally.plugins.common.scenarios.dummy.dummy


Dummy.dummy_timed_atomic_actions [Scenario]

Run some sleepy atomic actions for SLA atomic action tests.

Platform: default

Parameters
Argument Description
number_of_actions [ref] int number of atomic actions to create
sleep_factor [ref] int multiplier for number of seconds to sleep

Module: rally.plugins.common.scenarios.dummy.dummy


Dummy.failure [Scenario]

Raise errors in some iterations.

Platform: default

Parameters
Argument Description
sleep [ref] float iteration sleep time in seconds
from_iteration [ref] int iteration number which starts range of failed iterations
to_iteration [ref] int iteration number which ends range of failed iterations
each [ref] int cyclic number of iteration which actually raises an error in selected range. For example, each=3 will raise error in each 3rd iteration.

Module: rally.plugins.common.scenarios.dummy.dummy


HttpRequests.check_random_request [Scenario]

Executes random HTTP requests from provided list.

This scenario takes random url from list of requests, and raises exception if the response is not the expected response.

Platform: default

Parameters
Argument Description
requests [ref] List of request dicts
status_code [ref] Expected Response Code it will be used only if we doesn't specified it in request proper

Module: rally.plugins.common.scenarios.requests.http_requests


HttpRequests.check_request [Scenario]

Standard way for testing web services using HTTP requests.

This scenario is used to make request and check it with expected Response.

Platform: default

Parameters
Argument Description
url [ref] url for the Request object
method [ref] method for the Request object
status_code [ref] expected response code
kwargs [ref] optional additional request parameters

Module: rally.plugins.common.scenarios.requests.http_requests


Scenario Runner

Base class for all scenario runners.

Scenario runner is an entity that implements a certain strategy of launching scenarios plugins, e.g. running them continuously or periodically for a given number of times or seconds. These strategies should be implemented in subclasses of ScenarioRunner in the_run_scenario() method.

constant [Scenario Runner]

Creates constant load executing a scenario a specified number of times.

This runner will place a constant load on the cloud under test by executing each scenario iteration without pausing between iterations up to the number of times specified in the scenario config.

The concurrency parameter of the scenario config controls the number of concurrent iterations which execute during a single scenario in order to simulate the activities of multiple users placing load on the cloud under test.

Platform: default

The input of this plugin should be valid to the following JSONSchema
{
    "$schema": "http://json-schema.org/draft-04/schema", 
    "type": "object", 
    "properties": {
        "concurrency": {
            "type": "integer", 
            "description": "The number of parallel iteration executions.", 
            "minimum": 1
        }, 
        "max_cpu_count": {
            "type": "integer", 
            "description": "The maximum number of processes to create load from.", 
            "minimum": 1
        }, 
        "timeout": {
            "type": "number", 
            "description": "Operation's timeout."
        }, 
        "times": {
            "type": "integer", 
            "description": "Total number of iteration executions.", 
            "minimum": 1
        }
    }, 
    "additionalProperties": false
}

Module: rally.plugins.common.runners.constant


constant_for_duration [Scenario Runner]

Creates constant load executing a scenario for an interval of time.

This runner will place a constant load on the cloud under test by executing each scenario iteration without pausing between iterations until a specified interval of time has elapsed.

The concurrency parameter of the scenario config controls the number of concurrent iterations which execute during a single sceanario in order to simulate the activities of multiple users placing load on the cloud under test.

Platform: default

The input of this plugin should be valid to the following JSONSchema
{
    "$schema": "http://json-schema.org/draft-04/schema", 
    "type": "object", 
    "properties": {
        "concurrency": {
            "type": "integer", 
            "description": "The number of parallel iteration executions.", 
            "minimum": 1
        }, 
        "duration": {
            "type": "number", 
            "description": "The number of seconds during which to generate a load.", 
            "minimum": 0.0
        }, 
        "timeout": {
            "type": "number", 
            "description": "Operation's timeout.", 
            "minimum": 1
        }
    }, 
    "additionalProperties": false, 
    "required": [
        "duration"
    ]
}

Module: rally.plugins.common.runners.constant


rps [Scenario Runner]

Scenario runner that does the job with specified frequency.

Every single scenario iteration is executed with specified frequency (runs per second) in a pool of processes. The scenario will be launched for a fixed number of times in total (specified in the config).

An example of a rps scenario is booting 1 VM per second. This execution type is thus very helpful in understanding the maximal load that a certain cloud can handle.

Platform: default

The input of this plugin should be valid to the following JSONSchema
{
    "$schema": "http://json-schema.org/draft-04/schema", 
    "type": "object", 
    "properties": {
        "max_concurrency": {
            "type": "integer", 
            "minimum": 1
        }, 
        "max_cpu_count": {
            "type": "integer", 
            "minimum": 1
        }, 
        "rps": {
            "anyOf": [
                {
                    "type": "number", 
                    "description": "Generate constant requests per second during the whole workload.", 
                    "minimum": 0, 
                    "exclusiveMinimum": true
                }, 
                {
                    "type": "object", 
                    "description": "Increase requests per second for specified value each time after a certain number of seconds.", 
                    "properties": {
                        "duration": {
                            "type": "number", 
                            "minimum": 1
                        }, 
                        "end": {
                            "type": "number", 
                            "minimum": 1
                        }, 
                        "start": {
                            "type": "number", 
                            "minimum": 1
                        }, 
                        "step": {
                            "type": "number", 
                            "minimum": 1
                        }
                    }, 
                    "additionalProperties": false, 
                    "required": [
                        "start", 
                        "end", 
                        "step"
                    ]
                }
            ]
        }, 
        "timeout": {
            "type": "number"
        }, 
        "times": {
            "type": "integer", 
            "minimum": 1
        }
    }, 
    "additionalProperties": false, 
    "required": [
        "times", 
        "rps"
    ]
}

Module: rally.plugins.common.runners.rps


serial [Scenario Runner]

Scenario runner that executes scenarios serially.

Unlike scenario runners that execute in parallel, the serial scenario runner executes scenarios one-by-one in the same python interpreter process as Rally. This allows you to execute scenario without introducing any concurrent operations as well as interactively debug the scenario from the same command that you use to start Rally.

Platform: default

The input of this plugin should be valid to the following JSONSchema
{
    "$schema": "http://json-schema.org/draft-04/schema", 
    "type": "object", 
    "properties": {
        "times": {
            "type": "integer", 
            "minimum": 1
        }
    }, 
    "additionalProperties": true
}

Module: rally.plugins.common.runners.serial


Task Exporter

Plugin base for exporting tasks results to different systems&formats.

This type of plugins is designed to provide the way to present results in different formats and send them to the different systems.

  • To discover available plugins, call rally plugin list --plugin-base TaskExporter.
  • To export results of a task, call rally task export --uuid <task-uuid> --type <plugin-name> --to <dest>

elastic [Task Exporter]

Exports task results to the ElasticSearch 2.x, 5.x or 6.x clusters.

The exported data includes:

  • Task basic information such as title, description, status, deployment uuid, etc. See rally_task_v1_data index.
  • Workload information such as scenario name and configuration, runner type and configuration, time of the start load, success rate, sla details in case of errors, etc. See rally_workload_v1_data index.
  • Separate documents for all atomic actions. See rally_atomic_action_data_v1 index.

The destination can be a remote server. In this case specify it like:

https://elastic:changeme@example.com

Or we can dump documents to the file. The destination should look like:

/home/foo/bar.txt

In case of an empty destination, the http://localhost:9200 destination will be used.

Platform: default

Module: rally.plugins.common.exporters.elastic.exporter


html-static [Task Exporter]

Generates task report in HTML format with embedded JS/CSS.

Platform: default

Module: rally.plugins.common.exporters.html


html [Task Exporter]

Generates task report in HTML format.

Platform: default

Module: rally.plugins.common.exporters.html


json [Task Exporter]

Generates task report in JSON format.

Platform: default

Module: rally.plugins.common.exporters.json_exporter


junit-xml [Task Exporter]

Generates task report in JUnit-XML format.

An example of the report (All dates, numbers, names appearing in this example are fictitious. Any resemblance to real things is purely coincidental):

<testsuites>
  <!--Report is generated by Rally 0.10.0 at 2017-06-04T05:14:00-->
  <testsuite id="task-uu-ii-dd"
             errors="0"
             failures="1"
             skipped="0"
             tests="2"
             time="75.0"
             timestamp="2017-06-04T05:14:00">
    <testcase classname="CinderVolumes"
              name="list_volumes"
              id="workload-1-uuid"
              time="29.9695231915"
              timestamp="2017-06-04T05:14:44" />
    <testcase classname="NovaServers"
              name="list_keypairs"
              id="workload-2-uuid"
              time="5"
              timestamp="2017-06-04T05:15:15">
      <failure>ooops</failure>
    </testcase>
  </testsuite>
</testsuites>

Platform: default

Module: rally.plugins.common.exporters.junit


Validator

A base class for all validators.

args-spec [Validator]

Scenario arguments validator.

Platform: default

Module: rally.plugins.common.validators


check_constant [Validator]

Additional schema validation for constant runner.

Platform: default

Module: rally.plugins.common.runners.constant


check_rps [Validator]

Additional schema validation for rps runner.

Platform: default

Module: rally.plugins.common.runners.rps


enum [Validator]

Checks that parameter is in a list.

Ensure a parameter has the right value. This value need to be defined in a list.

Platform: default

Parameters
Argument Description
param_name [ref] Name of parameter to validate
values [ref] List of values accepted
missed [ref] Allow to accept optional parameter
case_insensitive [ref] Ignore case in enum values

Module: rally.plugins.common.validators


es_exporter_destination [Validator]

Validates the destination for ElasticSearch exporter.

In case when the destination is ElasticSearch cluster, the version of it should be 2. or 5.

Platform: default

Module: rally.plugins.common.exporters.elastic.exporter


file_exists [Validator]

Validator checks parameter is proper path to file with proper mode.

Ensure a file exists and can be accessed with the specified mode. Note that path to file will be expanded before access checking.

Platform: default

Parameters
Argument Description
param_name [ref] Name of parameter to validate
mode [ref] Access mode to test for. This should be one of: * os.F_OK (file exists) * os.R_OK (file is readable) * os.W_OK (file is writable) * os.X_OK (file is executable) If multiple modes are required they can be added, eg: mode=os.R_OK+os.W_OK
required [ref] Boolean indicating whether this argument is required.

Module: rally.plugins.common.validators


jsonschema [Validator]

JSON schema validator.

Platform: default

Module: rally.plugins.common.validators


map_keys [Validator]

Check that parameter contains specified keys.

Platform: default

Introduced in: 1.2.0

Parameters
Argument Description
param_name [ref] Name of parameter to validate
required [ref] List of all required keys
allowed [ref] List of all allowed keys
additional [ref] Whether additional keys are allowed. If list of allowed keys are specified, defaults to False, otherwise defaults to True
missed [ref] Allow to accept optional parameter

Module: rally.plugins.common.validators


number [Validator]

Checks that parameter is a number that pass specified condition.

Ensure a parameter is within the range [minval, maxval]. This is a closed interval so the end points are included.

Platform: default

Parameters
Argument Description
param_name [ref] Name of parameter to validate
minval [ref] Lower endpoint of valid interval
maxval [ref] Upper endpoint of valid interval
nullable [ref] Allow parameter not specified, or parameter=None
integer_only [ref] Only accept integers

Module: rally.plugins.common.validators


required_contexts [Validator]

Validator checks if required contexts are specified.

Platform: default

Parameters
Argument Description
contexts [ref] list of strings and tuples with context names that should be specified. Tuple represent 'at least one of the'.

Module: rally.plugins.common.validators


required_param_or_context [Validator]

Validator checks if required image is specified.

Platform: default

Parameters
Argument Description
param_name [ref] name of parameter
ctx_name [ref] name of context

Module: rally.plugins.common.validators


required_params [Validator]

Scenario required parameter validator.

This allows us to search required parameters in subdict of config.

Platform: default

Parameters
Argument Description
subdict [ref] sub-dict of "config" to search. if not defined - will search in "config"
params [ref] list of required parameters

Module: rally.plugins.common.validators


required_platform [Validator]

Validates specification of specified platform for the workload.

Platform: default

Parameters
Argument Description
platform [ref] name of the platform

Module: rally.common.validation


restricted_parameters [Validator]

Validates that parameters is not set.

Platform: default

Parameters
Argument Description
param_names [ref] parameter or parameters list to be validated.
subdict [ref] sub-dict of "config" to search for param_names. if not defined - will search in "config"

Module: rally.plugins.common.validators


Verification Component

Verification Reporter

Base class for all reporters for verifications.

html-static [Verification Reporter]

Generates verification report in HTML format with embedded JS/CSS.

Platform: default

Module: rally.plugins.common.verification.reporters


html [Verification Reporter]

Generates verification report in HTML format.

Platform: default

Module: rally.plugins.common.verification.reporters


json [Verification Reporter]

Generates verification report in JSON format.

An example of the report (All dates, numbers, names appearing in this example are fictitious. Any resemblance to real things is purely coincidental):

{"verifications": {
    "verification-uuid-1": {
        "status": "finished",
        "skipped": 1,
        "started_at": "2001-01-01T00:00:00",
        "finished_at": "2001-01-01T00:05:00",
        "tests_duration": 5,
        "run_args": {
            "pattern": "set=smoke",
            "xfail_list": {"some.test.TestCase.test_xfail":
                               "Some reason why it is expected."},
            "skip_list": {"some.test.TestCase.test_skipped":
                              "This test was skipped intentionally"},
        },
        "success": 1,
        "expected_failures": 1,
        "tests_count": 3,
        "failures": 0,
        "unexpected_success": 0
    },
    "verification-uuid-2": {
        "status": "finished",
        "skipped": 1,
        "started_at": "2002-01-01T00:00:00",
        "finished_at": "2002-01-01T00:05:00",
        "tests_duration": 5,
        "run_args": {
            "pattern": "set=smoke",
            "xfail_list": {"some.test.TestCase.test_xfail":
                               "Some reason why it is expected."},
            "skip_list": {"some.test.TestCase.test_skipped":
                              "This test was skipped intentionally"},
        },
        "success": 1,
        "expected_failures": 1,
        "tests_count": 3,
        "failures": 1,
        "unexpected_success": 0
    }
 },
 "tests": {
    "some.test.TestCase.test_foo[tag1,tag2]": {
        "name": "some.test.TestCase.test_foo",
        "tags": ["tag1","tag2"],
        "by_verification": {
            "verification-uuid-1": {
                "status": "success",
                "duration": "1.111"
            },
            "verification-uuid-2": {
                "status": "success",
                "duration": "22.222"
            }
        }
    },
    "some.test.TestCase.test_skipped[tag1]": {
        "name": "some.test.TestCase.test_skipped",
        "tags": ["tag1"],
        "by_verification": {
            "verification-uuid-1": {
                "status": "skipped",
                "duration": "0",
                "details": "Skipped until Bug: 666 is resolved."
            },
            "verification-uuid-2": {
                "status": "skipped",
                "duration": "0",
                "details": "Skipped until Bug: 666 is resolved."
            }
        }
    },
    "some.test.TestCase.test_xfail": {
        "name": "some.test.TestCase.test_xfail",
        "tags": [],
        "by_verification": {
            "verification-uuid-1": {
                "status": "xfail",
                "duration": "3",
                "details": "Some reason why it is expected.\n\n"
                    "Traceback (most recent call last): \n"
                    "  File "fake.py", line 13, in <module>\n"
                    "    yyy()\n"
                    "  File "fake.py", line 11, in yyy\n"
                    "    xxx()\n"
                    "  File "fake.py", line 8, in xxx\n"
                    "    bar()\n"
                    "  File "fake.py", line 5, in bar\n"
                    "    foo()\n"
                    "  File "fake.py", line 2, in foo\n"
                    "    raise Exception()\n"
                    "Exception"
            },
            "verification-uuid-2": {
                "status": "xfail",
                "duration": "3",
                "details": "Some reason why it is expected.\n\n"
                    "Traceback (most recent call last): \n"
                    "  File "fake.py", line 13, in <module>\n"
                    "    yyy()\n"
                    "  File "fake.py", line 11, in yyy\n"
                    "    xxx()\n"
                    "  File "fake.py", line 8, in xxx\n"
                    "    bar()\n"
                    "  File "fake.py", line 5, in bar\n"
                    "    foo()\n"
                    "  File "fake.py", line 2, in foo\n"
                    "    raise Exception()\n"
                    "Exception"
            }
        }
    },
    "some.test.TestCase.test_failed": {
        "name": "some.test.TestCase.test_failed",
        "tags": [],
        "by_verification": {
            "verification-uuid-2": {
                "status": "fail",
                "duration": "4",
                "details": "Some reason why it is expected.\n\n"
                    "Traceback (most recent call last): \n"
                    "  File "fake.py", line 13, in <module>\n"
                    "    yyy()\n"
                    "  File "fake.py", line 11, in yyy\n"
                    "    xxx()\n"
                    "  File "fake.py", line 8, in xxx\n"
                    "    bar()\n"
                    "  File "fake.py", line 5, in bar\n"
                    "    foo()\n"
                    "  File "fake.py", line 2, in foo\n"
                    "    raise Exception()\n"
                    "Exception"
                }
            }
        }
    }
}

Platform: default

Module: rally.plugins.common.verification.reporters


junit-xml [Verification Reporter]

Generates verification report in JUnit-XML format.

An example of the report (All dates, numbers, names appearing in this example are fictitious. Any resemblance to real things is purely coincidental):

<testsuites>
  <!--Report is generated by Rally 0.8.0 at 2002-01-01T00:00:00-->
  <testsuite id="verification-uuid-1"
             tests="9"
             time="1.111"
             errors="0"
             failures="3"
             skipped="0"
             timestamp="2001-01-01T00:00:00">
    <testcase classname="some.test.TestCase"
              name="test_foo"
              time="8"
              timestamp="2001-01-01T00:01:00" />
    <testcase classname="some.test.TestCase"
              name="test_skipped"
              time="0"
              timestamp="2001-01-01T00:02:00">
      <skipped>Skipped until Bug: 666 is resolved.</skipped>
    </testcase>
    <testcase classname="some.test.TestCase"
              name="test_xfail"
              time="3"
              timestamp="2001-01-01T00:03:00">
      <!--It is an expected failure due to: something-->
      <!--Traceback:
HEEELP-->
    </testcase>
    <testcase classname="some.test.TestCase"
              name="test_uxsuccess"
              time="3"
              timestamp="2001-01-01T00:04:00">
      <failure>
          It is an unexpected success. The test should fail due to:
          It should fail, I said!
      </failure>
    </testcase>
  </testsuite>
  <testsuite id="verification-uuid-2"
             tests="99"
             time="22.222"
             errors="0"
             failures="33"
             skipped="0"
             timestamp="2002-01-01T00:00:00">
    <testcase classname="some.test.TestCase"
              name="test_foo"
              time="8"
              timestamp="2001-02-01T00:01:00" />
    <testcase classname="some.test.TestCase"
              name="test_failed"
              time="8"
              timestamp="2001-02-01T00:02:00">
      <failure>HEEEEEEELP</failure>
    </testcase>
    <testcase classname="some.test.TestCase"
              name="test_skipped"
              time="0"
              timestamp="2001-02-01T00:03:00">
      <skipped>Skipped until Bug: 666 is resolved.</skipped>
    </testcase>
    <testcase classname="some.test.TestCase"
              name="test_xfail"
              time="4"
              timestamp="2001-02-01T00:04:00">
      <!--It is an expected failure due to: something-->
      <!--Traceback:
HEEELP-->
    </testcase>
  </testsuite>
</testsuites>

Platform: default

Module: rally.plugins.common.verification.reporters