Autotest Control files

This document will go over what is required to be in a control file for it to be accepted into SVN. The goal of this is to have control files that contain all necessary information for the frontend/the user to ascertain what the test does and in what ways it can be modified.

Naming your control files

Control files should always start with control.XXXXX. XXXXX is up to you and the code reviewer, the idea is for it to be short sweet and descriptive. e.g. For the hard reboot test 500 iterations test a decent name would be control.hard500

Variables

An overview of variables that should be considered required in any control file submitted to our repo.

Name	Description
*AUTHOR	Contact information for the person or group that wrote the test
DEPENDENCIES	What the test requires to run. Comma deliminated list e.g. 'CONSOLE'
*DOC	Description of the test including what arguments can be passed to it
EXPERIMENTAL	If this is set to True production servers will ignore the test
*NAME	The name of the test for the frontend
RUN_VERIFY	Whether or not the scheduler should run the verify stage, default True
SYNC_COUNT	Accepts a number >=1 (1 being the default)
*TIME	How long the test runs SHORT < 15m, MEDIUM < 4 hours, LONG > 4 hour
*TEST_CLASS	This describes the class for your the test belongs in. e.g. Kernel, Hardware
*TEST_CATEGORY	This describes the category for your tests e.g. Stress, Functional
*TEST_TYPE	What type of test: client, server

*Are required for test to be considered valid

AUTHOR

Cannot be empty

The name of either a group or a person who will be able to answer questions pertaining to the test should the development team not be able to fix bugs. With email address included

DEPENDENCIES

Cannot be empty

DEPENDENCIES is a comma deliminited list that describes the dependencies required to run the test. For example if a test requires conmux hard reset support DEPENDENCIES would be equal to 'POWER'. If the test requires multiple dependencies for example GCC and POWER it would be defined as:

DEPENDENCIES = 'POWER, GCC'

Current dependencies:

Dependency	Description
STANDARD	All autotest components no extra externals like conmux or power
GCC	Should be included if GCC is required for compiling tests
CONSOLE	Should be included if conmux is needed
POWER	Power management required

DOC

Cannot be empty

The doc string should be fairly verbose describing what is required for the test to be run successfully and any modifications that can be done to the test. Any arguments in your def execute() that can change the behavior of the test need to be listed here with their defaults and a description of what they do.

EXPERIMENTAL

DEFAULT = False

If this field is set the test import process for the frontend will ignore these tests for production autotest servers. This is useful for gettings tests checked in and tested in development servers without having to worry about them sneaking into production servers.

NAME

Cannot be empty

The name that the frontend will display, this is useful when you have multiple control files for the same test but with slight variations

RUN_VERIFY

DEFAULT = True

RUN_VERIFY is used to have the scheduler not run verify on a particular job when it is scheduling it.

SYNC_COUNT

DEFAULT = 1

SYNC_COUNT accepts a number >=1 (1 being the default). If it's 1, then it's a async test. If it's >1 it's sync for that number. For example if I have a test that requires exactly two machines SYNC_COUNT = 2. The scheduler will then find the maximum amount of machines from the job submitted that will run. For example if I submit a job with 23 machines, 22 will run and one will fail.

TIME

Cannot be empty

How long the test generally takes to run. This does not include the autotest setup time but just your individual test's time. TIME Description SHORT Test runs for a maximum of 15 minutes MEDIUM Test runs for less four hours LONG Test runs for longer four hours

TEST_CATEGORY

Cannot be empty

This is used to define the category your tests are a part of.

Examples of categories:

Functional
Stress

TEST_CLASS

Cannot be empty

This describes the class type of tests. This is useful if you have different different types of tests you want to filter on. If a test is added with a TEST_CLASS that does not exist the frontend should add that class.

Example tests classes

Kernel
Hardware

TEST_TYPE

Cannot be empty

This tells the frontend whether or not the test is a server side test or a client side test.

Valid options:

'server'
'client'

Example

TIME ='MEDIUM'

AUTHOR = 'Scott Zawalski < scott@xxx.com >'

NAME = 'Hard Reboot'

SYNC_COUNT = 1

TEST_TYPE = 'server'

TEST_CLASS = 'Hardware'

TEST_CATEGORY = 'Functional'

DEPENDENCIES = 'POWER, CONSOLE'

DOC = """\

Tests the reliability of platforms when rebooted. This test allows

you to do a hard reboot or a software reboot.

Args:

type: can be "soft" or "hard", default is "hard"

e.g. job.run_test('reboot', machine, type="soft")

This control file does a HARD reboot

"""

def run(machine):

job.run_test('reboot', machine, type="hard")

parallel_simple(run, machines)