Test data

A realistic load test needs more than every VU buying the same product with the same user name: different user accounts, different product ids, valid TC kimlik numbers, tokens received at login… This page covers where test data can come from in Spitfire, how CSV data files are uploaded and bound to a test, the fake data generators, passing values between steps, and where sensitive values such as passwords belong.

#What it is for

• The rows of the CSV files you upload on the Data files page are handed out to iterations: every iteration takes one row from each data binding.
• Rows can be given in order, at random or once each over the run; with several runners the rows are split between them.
• Syntaxes such as {{fake.tckn}}, {{fake.iban}}, {{fake.phone}} (fake data) make a new, algorithmically valid piece of Turkish test data on every use.
• A value taken from a step's response with an extractor is used in later steps.

#When to use it

• When every VU must log in with a different user (e-mail and password in a CSV).
• When different records must be read so the cache does not flatter the results (product or customer ids in a CSV).
• When endpoints with form validation (TCKN, IBAN, phone) must receive values that look valid (fake data).
• When every record must be used only once (e.g. single-use coupons: Unique mode).

#Data sources

For the full list of built-in values see Test editor.

#Uploading a CSV data file

1. Go to Data files in the left menu (Data files In Spitfire: /data-files).
2. Click Upload CSV at the top right. The Upload a data file dialog opens.
3. Choose the file in the CSV file field. Rules:
   • The first row is the header: column names are letters, digits or _ and do not start with a digit (email, password, customer_id). Non-ASCII letters, spaces and hyphens are not allowed.
   • A column name cannot appear twice; at most 100 columns.
   • There must be at least one data row after the header.
   • The file can be at most 20 MB.
   • The BOM in Excel's "CSV UTF-8" output is not a problem.
4. Type a name for the file in the Name field (the name shown in the list).
5. Click Upload CSV. The file appears in the list with its columns, row count, size and upload date.
6. The eye icon at the right of the row (Preview) shows the first 20 rows and Used by tests.

Data files page

Upload a data file dialog

Example users.csv:

email,password,customer_id
ayse.demir@example.com,Test1234!,100231
mehmet.kaya@example.com,Test1234!,100232
zeynep.yilmaz@example.com,Test1234!,100233

Preview dialog: the first 20 rows and the tests using the file

#Generating a data file

For a file with thousands of rows without using real customer data, use Data files → Generate data:

1. Click Generate data on the Data files page. The Generate test data dialog opens.
2. Type the file name in the Name field (e.g. customers).
3. Type the number of rows in the Rows field: 1 to 50,000.
4. Tick the value kinds you want in the table (TC kimlik no, IBAN (TR), Mobile phone, First name, Last name, Full name, E-mail, Province (il), District (ilçe), Address, Date) and change the Column name if needed. For Date, From and To can be set.
5. Click Generate and save. The file is added to the list; it is bound to a test like an uploaded CSV.

"The columns of one row describe one person: the e-mail is made from the name and the district belongs to the province." So the name, e-mail and address in one row are consistent with each other, while {{fake.*}} syntaxes make independent values on every use.

Generate test data dialog

#Binding a data file to a test

An uploaded file is attached to a test with a data binding. The binding's name is the name written in front of the columns in templates.

1. Open the test in the editor and switch to the General & variables tab.
2. Find the Data (CSV) section in the right column. If there are no files yet it says "Upload a CSV first:" with a Data files link.
3. Click Add a data binding. A row is added.
4. Type the binding's name in the Name field (e.g. users). Rules: letters, digits, _; not starting with a digit; unique in the test; the name fake cannot be used (it is reserved for fake data).
5. Pick the uploaded file in the File list.
6. Pick Sequential, Random or Unique in the Mode list (below).
7. Under the row, the syntaxes you can use are listed: {{users.email}}, {{users.password}}, {{users.customer_id}}.
8. Use them in steps, e.g. in the login body: {"email": "{{users.email}}", "password": "{{users.password}}"}.
9. Check with Try it, then save.

Data (CSV) section on the General & variables tab

The hint on screen states the most important rule: "Every iteration takes one row from each binding; all steps of that iteration see the same row." A scenario takes rows only from the bindings its own steps use; a scenario that never uses a binding consumes no rows from it.

In JSON (dataFileId is the file's id; the editor fills it in when you pick the file):

"data": [
  { "name": "users", "dataFileId": "8d3c1f6e-2b7a-4f1e-9c55-0a6b2e4d9f10", "mode": "unique" }
]

#Binding modes

#Several runners

When a run is spread over several runners:

• Sequential and Unique: each runner takes its own contiguous part of the file; no two runners use the same row. In Sequential mode, when there are more runners than rows, every runner uses the whole file; in Unique mode this does not happen (so rows do not repeat).
• Random: every runner gets the whole file.

#In comparative runs

In comparative (A/B) runs, Unique bindings are split into two equal halves for the two arms (A takes the first half, B the second; with an odd number of rows the last row is not used), so the arms never use the same row. With Sequential and Random bindings both arms read the whole file. In a sequential comparison, too, every turn takes its own equal slice of a Unique file.

#Fake data

Fake data is synthetic Turkish data generated on the runner during the test. It is written in any template field of a step and makes a new value on every use.

To copy one: expand Synthetic test data (fake.…) in a step card and click the button of the syntax you want (it is copied to the clipboard).

Fake data buttons in a step card

Example: a registration form body

{
  "tckn": "{{fake.tckn}}",
  "name": "{{fake.fullName}}",
  "email": "{{fake.email}}",
  "phone": "{{fake.phone}}",
  "iban": "{{fake.iban}}",
  "birthDate": "{{fake.date 1970-01-01 2000-12-31}}",
  "city": "{{fake.city}}"
}

Good to know:

• Writing {{fake.email}} twice in the same step gives two different e-mails. To use the same value in two places, either create a file with Generate data and bind it, or send the value in one step and extract it from the response.
• Giving an argument to a generator that takes none ({{fake.tckn 5}}) and an unknown name ({{fake.tc}}) are template errors; the error message lists the known names.
• A date range uses YYYY-MM-DD, and the end cannot be before the start.

#Passing values between steps

The way to use a value from a step's response in later steps is an extractor (details: Test editor).

Scope of values:

• Per VU: every VU has its own copy of the variables. A token one VU received does not pass to another VU.
• Across iterations: an extracted value stays in the VU's later iterations (until the next extraction changes it). That is how the token of a login step marked Run once per VU is used in all iterations.
• When not found: the value does not linger from the previous iteration; it falls back to the extractor's default, otherwise to the test's variable of the same name, otherwise to an empty string.
• Data row: all steps of an iteration see the same row; the next iteration takes a new row.

A typical flow:

1. login step (Run once per VU ticked): {{users.email}} and {{users.password}} in the body; Extract: token ← JSONPath $.access_token.
2. create_order step: header Authorization: Bearer {{token}}; Extract: orderId ← JSONPath $.id.
3. get_order step: URL {{base}}/api/orders/{{orderId}}.

In the Try it window each step shows its "Extracted" variables and the bottom shows the Final VU variables; check there that the values flow correctly.

Extracted values in the Try window

#Passwords and sensitive values

Spitfire has no separate secret vault for templates. It matters to know what goes where:

More recommendations:

• Values given with Change variables for this run in the Run dialog are recorded on the run and shown on the run page; don't put production secrets there either.
• Failed request samples mask query values that look like secrets and URL passwords, and never keep cookies. If responses can still contain personal data, set Options → Failed request samples → Off in the editor.
• If you test against a production system, mark the steps that change data (This request changes data on the target); starting a run then asks for an extra confirmation.

#Full example

Login with a user from the CSV, registration with fake data, and chaining with extractors:

{
  "version": 1,
  "name": "Customer registration and order",
  "variables": { "base": "https://shop.staging.example.com" },
  "data": [
    { "name": "users", "dataFileId": "8d3c1f6e-2b7a-4f1e-9c55-0a6b2e4d9f10", "mode": "sequential" }
  ],
  "options": {},
  "scenarios": [
    {
      "name": "register_and_order",
      "executor": { "type": "constant-vus", "vus": 20, "duration": "5m" },
      "steps": [
        {
          "id": "login",
          "name": "Login",
          "once": true,
          "protocol": "http",
          "request": {
            "method": "POST",
            "url": "{{base}}/api/login",
            "body": { "type": "json", "content": "{\"email\": \"{{users.email}}\", \"password\": \"{{users.password}}\"}" }
          },
          "extract": [ { "var": "token", "from": "jsonpath", "expr": "$.access_token" } ],
          "checks": [ { "type": "status", "op": "eq", "value": 200 } ]
        },
        {
          "id": "add_customer",
          "name": "Add customer",
          "protocol": "http",
          "request": {
            "method": "POST",
            "url": "{{base}}/api/customers",
            "headers": [ { "key": "Authorization", "value": "Bearer {{token}}" } ],
            "body": { "type": "json", "content": "{\"tckn\": \"{{fake.tckn}}\", \"name\": \"{{fake.fullName}}\", \"phone\": \"{{fake.phone}}\", \"iban\": \"{{fake.iban}}\"}" },
            "modifiesData": true
          },
          "extract": [ { "var": "customerId", "from": "jsonpath", "expr": "$.id" } ],
          "checks": [ { "type": "status", "op": "in", "value": [200, 201] } ]
        },
        {
          "id": "get_customer",
          "name": "Read customer",
          "protocol": "http",
          "request": {
            "method": "GET",
            "url": "{{base}}/api/customers/{{customerId}}",
            "headers": [ { "key": "Authorization", "value": "Bearer {{token}}" } ]
          },
          "checks": [
            { "type": "status", "op": "eq", "value": 200 },
            { "type": "jsonPath", "path": "$.id", "op": "exists" }
          ],
          "thinkTime": { "min": "1s", "max": "2s" }
        }
      ]
    }
  ],
  "thresholds": [
    { "metric": "req_failed", "expr": "rate<0.01" },
    { "metric": "req_duration", "expr": "p(95)<600" }
  ]
}

In SQL steps values go into the Parameters, not into the query; data columns and fake data can be parameter values:

"sql": {
  "query": "SELECT id, status FROM orders WHERE customer_id = $1 ORDER BY created_at DESC LIMIT 10",
  "params": [ "{{users.customer_id}}" ]
}

SQL step form: query and parameters

#Common problems

#Related pages

• Test editor: extractors, checks and template rules.
• Load model: understanding iteration counts and the split across runners.
• Runs and results: changing variables per run.
• Data files In Spitfire: /data-files · Connections In Spitfire: /connections