Simplified block diagram
For the same of clarity, some things are simplified here, but it gives you a good idea of the overall layout.
User Interface
We try to let you do everything both through the web, and from a command line. Allows you to:
Manage jobs - create, monitor, abort, etc.
- Manage client hosts - add, destroy, monitor, etc.
- Look at results.
We use Django as a database abstraction layer, though at the moment, the results information doesn't use it.
The frontend injects jobs into the server by creating records into the frontend database (see below)
Databases and Results repository
There are two main databases (which use mysql), and results repository (which is just a directory tree).
- A frontend database that holds information on all jobs, clients (test machines), users and tests. This controls the operation of the server.
- The results repository - there's a directory tree with a subdirectory for each job containing all the logged data on that job.
- A backend database that holds indexed information about job results.
For more information on how we handle jobs results, see here.
Server
The server consists of two main parts. All its state is stored in the databases described above.
- The scheduler (monitor_db) - chooses the jobs from the database to run. It's input is the database, pretty much all it does is start autoserv processes to service requests.
- There is normally one dispatcher process per machine
- Client side jobs are run asyncronously (as client machines become available)
- Server side jobs are run syncronously (ie we wait for all clients before commencing)
The dispatcher (autoserv): the server manages clients via autoserv processes - there will be one autoserv process per running job. Each autoserv process:
- controls and monitors one or more clients
- verifies clients are working properly, and if it fails verification, attempts to repair it
manages the execution of a job
- updates the autotest software on each client before commencing work.
The mysql database can live on a different machine than the dispatcher. There can be multiple dispatchers to spread the workload, though each can service a few thousand clients, so this is not normally necessary.
Client
The client does most of the work of running a job; this can be invoked manually on the client, or via the server - see RunningTests
Analysis backend / test.kernel.org
A set of analysis scripts that:
Parse the results and populate the mysql database with data.
- Dynamically query the mysql database for data
May also provide links back to the results repository for data drill-down, if available via the web.
Job Flow Diagram
