Assessment FAQ

They’re a guide for planning. One timer runs for the whole assessment.

When time runs out or you select Finish, every challenge you opened is submitted with its latest saved work: your chosen answer, your saved response, or the files saved in your automation workspace. Challenges you never opened count as not submitted. On a question, Submit answer is final right away and asks for no confirmation.

No. Stay in the assessment until you finish. Switching to any other browser tab or program, including your notes, email, chat, documentation or this FAQ, is recorded as leaving the assessment. Moving between the assessment page and the second window it can open for the application you’re testing doesn’t count. The Privacy Notice lists everything the assessment records.

Only in the Live app, the view in your workspace that shows the application (called Live API when the application is only an API). Addresses such as localhost:3000 don’t work in your own browser. To inspect it, use the developer tools of the browser inside the Live app; opening developer tools in your own browser counts as leaving.

In the Live app, selecting text doesn’t copy it. Copy with the keyboard (Control+C, or Command+C on a Mac) or the right-click menu, and allow your browser’s clipboard prompt when it asks.

You pasted text the assessment didn’t see you copy, after you had left it at least once. The paste is recorded, and the hiring team sees it as text pasted after leaving. Copies made in the editor’s terminal aren’t seen, so pasting them can show this too. The Privacy Notice explains what is recorded.

It starts the challenge’s environment again. The application you’re testing comes back with only its starting data, so data you added is gone. In an automation challenge, your files come back as they were at the last save to our servers. Each challenge’s environment can be restarted up to 3 times.

There’s no reset. Undo your own changes with the application’s own features where it has them: for example, delete an order you created through its API, or remove an item you added to the cart. Plan your test data with that in mind.

Select Open Live app in a new window, the icon just after Live app in the bar at the top of your workspace, and allow pop-ups if your browser asks. Reloading the page or switching challenges closes that window; open it again the same way.

No. The SQL editor is read-only: you can query the PostgreSQL database of the application you’re testing, but not change it, so add or remove data through the application or its API.

Everything in your workspace except these folders and files, wherever they are:

• node_modules
• .git
• dist
• build
• target
• .m2
• .gradle
• .npm
• .yarn
• .pnpm-store
• __pycache__
• .cache
• .pytest_cache
• .venv
• venv
• playwright-report
• test-results
• coverage
• allure-report
• allure-results
• .idea
• .vscode
• .DS_Store
• Thumbs.db

Files you put in them aren’t saved or submitted. The INSTRUCTIONS.md and README.md you start with, and your starter’s package-lock.json or requirements.lock, aren’t submitted either, and changes to them aren’t kept.

At most 100 kilobytes and 100 files, counting only what is submitted. Your work keeps saving while you are over, but you can’t submit, and a challenge still over the limit when the assessment ends isn’t evaluated.

Every few minutes, not each time you save in the editor. If the environment restarts, your files come back as they were at the last of those saves.

No. Everything the starter needs is already installed.

Once, with your project’s standard test command (npm test, npx playwright test, pytest or mvn test, depending on the starter), against a freshly started copy of the application you’re testing. The whole run is limited to 2 minutes in API Test Automation and 5 minutes in Web Test Automation. Data you added by hand and settings you made in the terminal, such as environment variables, aren’t there.

The evaluation finds test files by their names, using the same pattern as the starter’s example test. Name yours like it: tests/login.spec.ts next to tests/demo.spec.ts, test_login.py next to test_demo.py, or LoginTest.java next to DemoTest.java. A file named another way may not be run, even if it runs when you give its name to the command, as in pytest checks.py. With Selenium or Axios in TypeScript, one type error in your code stops every test.

They’re evaluated on a 1,920 × 1,080 screen. To work at that size, choose Full HD in the Live app’s Screen size box.

In the Live app, not the Code editor.

The challenge’s environment closes and anything still running in it stops. You can’t reopen it.

No. Only the text in Your response is submitted. Write down each request or query you ran and what it returned; you can’t attach screenshots.

Up to 20,480 characters. Over that, you can’t save, submit or move to another challenge.

Your latest changes are only on this device. Wait for “Saved” before you switch challenges or close the page. If you choose Switch anyway, those changes are evaluated only if they save later.

Email QAVetted support at [email protected] and say what happened. Writing the email is recorded as leaving the assessment. If you can’t continue, also tell the hiring team, which decides what happens next.

Not yet. A Live Interview has no Finish; your interviewer ends the session.