Everything SetBenchmark does, chapter by chapter: what each feature is for and how to use it, step by step. Features that aren’t fully working on the live site yet are marked in amber or red, with the reason and the steps to switch them on once the account, key or licence is bought.
Updated 21 November 2026, from the guide and the live site’s settings.
No feature matches.
1. Getting started
How to sign in, join your team, and find your way around.
The SetBenchmark website and finding your workspace Ready
www.setbenchmark.com is SetBenchmark’s public website: what the product does, pricing, skill tests, the blog and contact details. It is not where your team works. (Typing setbenchmark.com takes you there too.)
Full details
The app itself is at app.setbenchmark.com: signing in, starting a free trial, this guide, and badge and status pages. Older links on setbenchmark.com, such as setbenchmark.com/login or setbenchmark.com/guide/, still work and take you to the same page on app.setbenchmark.com.
To get to your company, click “Log in” on the website (or open app.setbenchmark.com/login), then “Sign in with your email”: after you sign in you see every company your email belongs to. Or open your company’s own address directly, such as kombee.setbenchmark.com. Candidates never need the homepage: their invitation email has their own link.
In the guide: Getting started → The SetBenchmark website and finding your workspace
Starting a free trial for your company Ready
The trial is free for 14 days, with 10 assessment attempts and every module switched on (AI use is capped). No card is needed. When it ends, the company moves to the free plan unless you choose a paid one under Settings → Billing, which shows the trial and its end date until then.
Full details and steps
A company trial needs a work email. Personal addresses such as Gmail, Yahoo, Outlook.com, iCloud or Proton can’t start one, because your teammates join by your company’s email domain. If you sign up with one, you are told straight away after confirming your email, with a button to sign out and sign up again with your work email.
One person can start at most 3 trials in 30 days. If “Start free trial” is missing, sign-up is switched off for now: ask Kombee to set up your company.
On the Pricing page, plans priced only in rupees show their rupee price with “billed in INR” when you look at another currency.
How to do it:
- On www.setbenchmark.com click “Start free trial” or “Sign up” (it opens app.setbenchmark.com/signup). You can also open app.setbenchmark.com/signup directly.
- Click “Create your account”. Enter your work email (you@yourcompany.com) and choose a password of at least 10 characters, or continue with your work Google or Microsoft account.
- Type the 6-digit code we email you.
- Back on SetBenchmark, “Set up your workspace”: your name (shown to your team), your company name (shown to candidates), your workspace address (filled in from the name; it says “✓ Available” or suggests a free one), country, time zone, where your data is kept (India for now), how many engineers you hire a year, and tick the Terms. Click “Create workspace”.
- You are taken straight into your new workspace at its own address, for example yourcompany.setbenchmark.com, already signed in, on “Get started”: pick the role you’re hiring for and your first test is published in a click (see “Get started: your first test in a minute”). Home shows “Free trial: 14 days left”. Next time, sign in at that address.
In the guide: Getting started → Starting a free trial for your company
Get started: your first test in a minute Ready
A new company opens on “Get started” (also on Home, under Quick start). It has four steps, each done on the page.
Full details
1. Make your first test: click the role you are hiring for, such as “Backend developer (Node.js)”, “Python developer” or “Data analyst (SQL)”. There are 14: Node.js, React, full-stack JavaScript, JavaScript, Python, Java, Go, SQL, data science (with a notebook), DevOps, AWS, QA, data structures and algorithms, and system design (with a whiteboard task). A ready test from SetBenchmark’s reviewed question library is made and published in a second or two, with its link to copy and send to candidates. Each role shows how many questions it gets and how long the test is. Hiring for something else? “Paste a job description” makes a test with AI instead (about 20 seconds). Change anything in the test later.
2. See a finished result: the first time, a sample candidate takes your starter test, and within a minute or two “Open the sample candidate’s result” shows what a marked result looks like.
3. Try your test as a candidate yourself: “Take the test as a candidate” opens it on the candidate site in a new tab, as your candidates will see it.
4. Bring in your team: invite recruiters, hiring managers and interviewers.
Once your first test is published, the page says how long it took from signing up.
Tip: Sample candidates (the example and your own tries) are marked “sample”: they use no credit or trial attempt and aren’t counted anywhere. Delete them whenever you like.
In the guide: Getting started → Get started: your first test in a minute
Signing in Partly ready
Open your company’s own address, for example northwind.setbenchmark.com. The form is on the right of the page, with your company’s name (and logo, if it has one) above the sign-in buttons.
Full details and steps
Choose how to sign in. Your admin decides which ways are allowed: your company’s single sign-on, Microsoft, Google or your email address.
How to do it:
- Open your company’s address.
- Click the way you sign in, for example “Continue with Google”.
- Follow the steps on the next screen. You come back to SetBenchmark signed in.
Tip: Candidates never sign in here. They use the link in their invitation email.
Tip: You are asked which account to use every time: Google and Microsoft show their list of accounts, and email sign-in asks again. Pick the work account your admin added. “No active account for this company” means the account you picked isn’t on this company’s team.
Tip: Forgot your password? Choose “Sign in with email”, type your address, then “Forgot password?”. The reset email comes from the sign-in service. Finish within 30 minutes, in the same browser. If the reset link opens in another browser, the new password is still saved: go back to your company’s address and sign in again.
Tip: If you are signed out after a while without activity, simply sign in again. Your work is saved.
What isn’t finished
Email and Google sign-in work. Microsoft sign-in is hidden until its app is registered.
Waiting for: Kombee IT. Microsoft sign-in needs an app registered in Kombee’s own Microsoft (Entra) organisation.
How to switch it on:
- Kombee IT registers an app in Entra (Microsoft Entra admin centre → App registrations → New registration), with the redirect URI shown in WorkOS → Authentication → Providers → Microsoft.
- Create a client secret and enter the Client ID and secret in that WorkOS Microsoft provider, then enable it.
- On the server, set SOCIAL_SIGN_IN=google,microsoft in /etc/setbenchmark.env and restart the API.
In the guide: Getting started → Signing in
Joining your team from an invitation Ready
When an admin adds you to the team (as staff or as an employee for skills tests), you get an invitation email. Its link opens this page, with your company’s look and your email address filled in.
Full details and steps
How to do it:
- Open the link in the invitation email.
- First time? Click “Create your account”, choose a password and confirm your email. Already have an account? Click “I already have an account: sign in”.
- Use the same email address the invitation was sent to. If you see “No active account for this company for …”, you signed in with a different address: sign out of it and try again with the invited one.
In the guide: Getting started → Joining your team from an invitation
The home page Ready
After signing in you land on the home page. It greets you by name and says how many items wait for a person to review. Nothing on it has been decided automatically.
Full details
Four numbers run along the top: active assessments (and drafts), candidates part-way through a test, items awaiting your review, and integrity notes to review.
“Needs your attention” lists, for example: requests from candidates, appeals, integrity notes to review, candidates to decide on (everyone who has finished a test and has no decision yet), and written answers to mark. Each line has a button that takes you straight there.
“Active assessments” shows how far invited candidates have got in each test. On the right are today’s interviews, “Usage this month” (for people who may see usage, with the plan’s limits) and “Decisions”: how many candidates were advanced, how many decisions people made, and how often they agreed with the AI.
Tip: Nothing on this page is decided by software. Every item waits for a person.
In the guide: Getting started → The home page
Finding your way: the menu, search and the top bar Ready
The left menu is in this order: Home, Assessments, Candidates, Interviews, Question library, Skills and roles, Analytics, Employee skills, Integrations and API, Settings. You only see what your role and your company’s plan include. Grading queue, Integrity review and Appeals sit with the work they belong to.
Full details
At the top of the menu is your company’s name. Click it and choose “Switch company” if you work for more than one company.
At the bottom is your card: your name, initials and role, with “My account”. People who manage API keys also see whether sandbox mode is on, with a link to it.
The top bar has a search box: as you type a name, a test title or a question, a few matches appear under it (use the arrow keys and Enter, or click one). Press Enter without choosing one to see all matching candidates, assessments and questions on one page. The bell (your notifications) shows how many items wait for you; click it to open “Waiting for you”, a page with every item (Home shows the first ten). And “+ New assessment” (for people who make tests) starts a new test.
In the guide: Getting started → Finding your way: the menu, search and the top bar
Waiting for you: everything that needs a person Ready
Click the bell in the top bar to open “Waiting for you”. It lists everything that is waiting for someone: results to decide, written answers to mark, integrity reviews, appeals and candidate requests. The number on the bell is how many there are; Home shows the first ten.
Full details
Each row says who it is about, what it concerns and what is needed. Click “Review” or “Mark” to go straight to it. Answers to mark show “Names hidden”, so marking stays blind. Nothing on this page is decided automatically: each item waits for a person.
In the guide: Getting started → Waiting for you: everything that needs a person
Home for interviewers Ready
Interviewers see their own work: answers waiting to be marked and today’s interviews.
In the guide: Getting started → Home for interviewers
Home for employees Ready
Employees see links to their skill tests and their skills.
In the guide: Getting started → Home for employees
Your account Ready
Click “My account” at the bottom left. Add your full name, see your role and time zone, and see where you are signed in. “Sign out everywhere else” signs out every other device, for example a computer you no longer use.
In the guide: Getting started → Your account
The demo account: view as every role Ready
A demo company can have one demo account. It signs in with one email and password like anyone else, and then sees a “View as” menu at the bottom left, above its name.
Full details and steps
Pick a role and click “Switch”: Company admin, Recruiter, Hiring manager, Interviewer, HR manager or Employee opens the product as that company’s sample person with that role, so you see exactly their menus and pages. Choose Company admin to switch back.
Candidate makes a new sample candidate, invites them to the company’s oldest published test and opens the candidate site, as if you clicked the link in their email. Your staff screen stays signed in: go back to the company’s address to carry on.
Super admin (platform console) appears when the demo email is also on the Kombee staff list. It opens the platform console on the main address, which asks you to sign in there with the same email and password. The menu says so beside the role. That sign-in is separate and doesn’t last as long as your company one, so you may be asked again.
Only Kombee can make a demo account, from the server (pnpm db:role-switcher). Nobody can turn it on from the screens or the API.
How to do it:
- Sign in with the demo email and password.
- At the bottom left, open “View as” and pick a role.
- Click “Switch”.
- To come back, pick “Company admin” and click “Switch”.
In the guide: Getting started → The demo account: view as every role
2. Tests
Build a test, publish it, invite candidates, and follow the results.
The list of tests Ready
Click “Assessments” in the left menu. Tests are split into three tabs: Active (published), Drafts and Closed.
Full details
For each test you see how many candidates were invited, how many finished, and how many answers wait for a person to mark.
Tip: Use “Starred”, “Mine”, “My teams” and the labels to find tests quickly. Click the star next to a test to keep it at hand.
In the guide: Tests → The list of tests
Drafts Ready
A draft can still change. Publishing locks it, so every candidate gets exactly the same test.
In the guide: Tests → Drafts
Making a new test Ready
Click “+ New assessment” in the top bar, or “New test” on the Assessments page. The page asks “How do you want to start?” and offers five ways:
Full details and steps
1. A test in a minute: paste a job description, look over the test the AI makes, publish it and share one link (only if your plan includes AI; see “A test in a minute” below).
2. From a job description: paste or upload the job advert and the AI suggests a test for you to check (only if your plan includes AI; see the next part).
3. From a template: ready-made tests per role, such as “Backend engineer (Java)”, with timed sections filled with suitable questions from the library. Use the category buttons to narrow the list.
4. Duplicate an existing one: a new draft with the same questions, sections and settings as an earlier test. The original is not changed.
5. Blank test: give it a title and length, then add questions yourself.
How to do it:
- Click “+ New assessment”.
- Pick a way to start. For a template click “Use template”; to copy, choose the test and click “Duplicate”; for a blank test fill in the form and click “Create the test”.
- The new draft opens. Check the questions before publishing.
Tip: If the template could not fill a section from your library, the draft tells you what is missing.
Tip: Your company’s hiring policies apply to every way of starting.
In the guide: Tests → Making a new test
A test in a minute (quick start) Ready
The quickest way to get going. Click “A test in a minute” on Home (under Quick start) or on the “How do you want to start?” page. Everything happens on one screen.
Full details and steps
Type the job title and paste the job description, then click “Make my test”. The AI picks the skills and levels and fills a test from the published questions in your library. This takes about 20 seconds. The test is named after the job title, which is what candidates see.
You then see the test: its title, length and every question. Skills the library had no question for are listed. Nothing has gone out yet: click “Change it first” to open it in the builder, or “Publish and get the link”.
Publishing makes a link anyone can use to take the test: copy it into an email or your job post. Candidates give their name and email first. Open the test afterwards to add a password or closing date to the link, invite people by email instead, or see who has taken it.
How to do it:
- Click “A test in a minute”.
- Type the job title and paste the job description. Click “Make my test”.
- Look over the questions. Click “Publish and get the link”.
- Click “Copy link” and send it to your candidates.
Tip: If your library has nothing for the job’s skills, the screen says so and doesn’t publish an empty test: write or import questions first.
Tip: For more control (swapping questions, choosing sections), use “From a job description” instead.
In the guide: Tests → A test in a minute (quick start)
A test from a job description Ready
Choose “From a job description”. Type the role title, then paste the job description or click “Upload PDF or DOCX” (a PDF, Word or text file up to 5 MB). Check the text, then click “Suggest a test”. This can take a minute.
Full details
You then see the role and skills the AI found, with the level for each, and the closest role profile. Skills shown with a dashed border are not in your skills list yet. If the job description uses wording that could put applicants off for reasons unrelated to the job, a yellow note says so.
On the right is the suggested test, in sections with minutes and whether AI help is allowed. Each question says why it was suggested. Untick questions you don’t want, or click “Swap” to get another published question on the same skill. Optional questions start unticked.
Tick the two checks (the questions fit the role, and the time is fair), give the test a title and click “Open in builder”. Nothing is saved until then, and nothing is published: it opens as a draft.
Tip: The AI only picks questions already published in your library. What it couldn’t fill is listed, so you can write those questions.
Tip: Reading a PDF uses the AI and counts in your AI usage; Word and text files don’t.
In the guide: Tests → A test from a job description
Recording a spoken answer in an Explain step Ready
In an Explain step, if a speech service is switched on, candidates can record their written answer for up to 3 minutes. Their words appear in the answer box for them to check, and a person marks it like any written answer. The recording isn’t kept.
In the guide: Tests → Recording a spoken answer in an Explain step
Steps on a real-work task Ready
On a test with sections, each section can be marked as a step: Debug, Build, Optimise or Explain. Choose it under “Step” when you edit the sections. The test page then shows which step each section is.
In the guide: Tests → Steps on a real-work task
Sections, settings and monitoring Ready
A test can be split into sections, for example “Fundamentals”, “Coding” and “Design”. Each section can have its own time; candidates then do the sections one after another and can’t go back.
Full details
A section can also have its own AI rule (AI allowed or not), and can give each candidate only some of its questions, picked at random. This makes answers harder to share.
Under “Questions and scoring” you can hide question titles and points from candidates, take points off for wrong multiple-choice answers (negative marking), allow retakes, let candidates use the built-in AI assistant, and ask for details before they start (for example their notice period).
Under “Monitoring” you choose what is checked during the test: focus and shortcut keys, the camera, screen sharing, one screen only. With the camera check you can also keep one photo of the candidate as the test starts, shown to the reviewer; it is never compared with other photos. To check who the candidate is, tick “Identity check” instead: an ID and a live selfie before the test, and face matching during it (see “Identity check: ID, live selfie and voice”). Candidates are told before they start and must agree.
Tip: Candidates who agree to adjustments (for example extra time) get that time in every section.
In the guide: Tests → Sections, settings and monitoring
More test options Ready
Under “Moving between sections”: with timed sections, tick “Carry unused time over” and a candidate who finishes a section early gets the time left added to the next one. Sections without their own time can still be taken in turn with no going back (“Take sections in turn”), or shuffled so each candidate gets them in a different order (“Shuffle the sections”).
Full details
“Watermark with the candidate’s email” shows their email faintly across every question, so a shared screenshot shows whose test it came from.
“Let candidates check code against hidden tests” adds a “Check all tests” button to coding questions: candidates see how many tests pass, never the hidden inputs or answers.
“Declaration” is your own short text, for example “I won’t copy code or use outside help”. Candidates must tick “I agree” before they can start.
Under “Results by email”, add up to 10 email addresses. Each gets the score and a link when a candidate’s result is ready. Tick “Only results at or above the pass mark” to hear only about candidates who pass.
Tip: Your company can fix some of these for every test in Settings → Hiring policies. A fixed setting is greyed out on the test.
In the guide: Tests → More test options
The test page Ready
Open a test to see everything about it on one page:
Full details
• Results: how many finished, the average score, how scores spread, which skills and which questions were hardest, and how much of their time candidates used.
• Sections and questions. In a draft you can move a question up or down and change its section.
• Candidates invited, and what each is doing.
• “Invite more” at the top right: invite a candidate or a list of people, and the shareable link.
• Feedback candidates gave about the test (anonymous).
To download the results, choose CSV, Excel or PDF next to the title. “Columns” lets you pick what the download has, for example the email, the minutes each candidate used, or a score for every question. Your choice is remembered on this computer.
In the guide: Tests → The test page
Inviting a candidate Ready
Full details and steps
How to do it:
- On the test page, click “Invite more” at the top right. Three tabs give the ways to invite: One person, A list (paste or upload up to 100 people) and Shareable link. On One person, choose the candidate; nobody is picked for you, and the form clears after each invite so a note can’t go to the next person. One person lists the candidates you have already added, so it only appears once you have some: add the candidate first under Candidates → “Add a candidate”, or use A list.
- Check the email address and how many days the test stays open.
- Add a personal note if you like. Click “Preview email” to see exactly what they will get.
- Choose “Extra time” if this candidate needs more time (for example +25%). They get it in every part of the test.
- Click “Invite”. The candidate gets an email with their own link. The link works once, until the test closes, and the email gives that date.
Tip: How long the invitation link works: once, until the test closes (the “Open for (days)” you chose). If it stops working, send a new link from the candidate’s page (“Candidate link”); a new link works for 72 hours.
Tip: To invite a list of people at once, open the “A list” tab. Paste one person a line (name, email), or choose a spreadsheet saved as CSV. Up to 100 at a time; people already invited are skipped, and lines without a valid email are listed so you can fix them.
Tip: To invite many candidates without names first, use “Make a link” on the “Shareable link” tab. Anyone with the link gives their name and email, then starts. You can add a password, a closing date and a limit. You also get a button to put on your careers page.
In the guide: Tests → Inviting a candidate
Helping a candidate: adjustments and extra time Ready
Candidates can ask for an adjustment before they start, such as extra time or screen-reader support. Their request shows next to their name, and on the home page.
Full details and steps
If something goes wrong on our side during a test, click “Give extra time” next to that candidate. Write a short note; the candidate sees it on their screen and their timer goes up.
How to do it:
- Read the request next to the candidate’s name.
- Click “Adjust” to give extra time or switch off a check, if you agree.
- Reply to the candidate, then click “Mark as answered”.
In the guide: Tests → Helping a candidate: adjustments and extra time
Campus drives: eligibility and results by college Ready
In the test’s settings, under “Ask candidates before they start”, add the details a drive needs, for example College (text), Graduation year (choose one), Percentage (number) and Any backlogs (yes or no). Publish the test and share its link: students register with their name, email and these details.
Full details
To limit who can take it, untick the answers that aren’t eligible under “Who can take the test” (for example only 2026 graduates, or no backlogs), or set a lowest or highest value on a number detail (for example a percentage of at least 60). A student outside the rule can’t start and is told why.
On the test page, the Results card then shows “By college” (and by every other detail): how many finished from each, their average and best score. The same college typed in a different case or spacing counts once.
In the guide: Tests → Campus drives: eligibility and results by college
Task catalogue: ready-made tasks Ready
Click “Task catalogue” for ready-made kinds of task, each with its platform, skills, level, time and how it’s marked. “Use this” makes a draft question from it for you to change and send for review.
Full details
Ready to use: SAP ABAP, Salesforce, ServiceNow, Workday integrations, DevOps and cloud (OpenTofu/Terraform, Kubernetes, AWS), Java, data science notebooks, prompt engineering, and senior reviews of AI-written pull requests (see the next sections). Tasks that need the vendor’s own system show “Needs a vendor licence” (a Salesforce scratch org, SAP BTP, a ServiceNow instance, a Workday tenant), and running Terraform against the real provider, a live Kubernetes cluster and Spark show “Needs bigger sandboxes”: those can’t be used yet.
In the guide: Tests → Task catalogue: ready-made tasks
Assessments for platform developers and consultants. Most IT staffing demand is for platform roles, not coding puzzles. Each of these is a real small project the candidate fixes in the browser and checks with “Run tests”, marked by hidden grading tests. They run in SetBenchmark’s own sandbox, with no internet and no vendor account:
Full details
SAP ABAP: an ABAP class and ABAP Unit tests, run by the open-source abaplint transpiler (not an SAP system: classes, internal tables and ABAP Unit, no database tables, CDS views or RAP). Salesforce: a Lightning Web Component tested with Salesforce’s own Jest set-up, plus Apex that a reviewer marks with a rubric (Apex needs a Salesforce org to run). ServiceNow developers: a Script Include run against a simulated GlideRecord and gs, so queries in loops show up. Workday: an outbound integration transform in XSLT 3.0, run with Saxon as Workday Studio does. DevOps and cloud: an OpenTofu/Terraform module checked as written, as policy tools such as Checkov do, Kubernetes manifests checked with kubeconform for Kubernetes 1.33, and a Python AWS audit tested against moto, a simulated AWS. Java: JUnit 5 with Mockito, AssertJ and Jackson.
In the repository task editor these appear under “Language and test command”, so you can also build your own task on any of them, including from a client’s code.
Tip: Nothing runs in a vendor’s system, so tasks that need one (Salesforce flows, SAP CDS views, ServiceNow ACLs, Workday business processes) are listed as needing a vendor licence.
What isn’t finished
Waiting for: Bigger sandbox server. All eight tasks run and are marked in the sandbox (checked on the live sandbox on 19 Nov). SAP, Salesforce, ServiceNow and Workday run on open-source runners, not on the vendors’ own systems. The Terraform task checks the configuration but doesn’t run a full “terraform test”, which needs a bigger sandbox machine.
How to switch it on:
- For full Terraform runs: approve a bigger sandbox machine (e2-standard-4) and change sandbox_machine_type in infra/gcp/terraform.tfvars, then terraform apply.
- For tasks on real vendor systems: buy developer/sandbox licences from SAP, Salesforce or ServiceNow and share the logins.
In the guide: Tests → Platform tasks: SAP ABAP, Salesforce, ServiceNow, Workday, DevOps and cloud
Senior reviews of AI-written pull requests Ready
For senior hires, the catalogue has four pull requests an AI assistant might open, each across several files with a title, a description and a CI result: bulk refunds in a Node.js payments service, a Terraform database upgrade, a Java Spring order endpoint and a Salesforce Apex trigger. Each has 6 to 8 planted problems, including security, reliability (retries, transactions, idempotency) and design.
Full details
Candidates see the diff as one list of lines, coloured for added and removed lines, with the file names. They comment on lines, give a severity, write a summary for the author and approve or request changes. The planted problems are marked automatically for 70% of the points; a reviewer marks the rest with the rubric (judging the change as a whole, rollout risk, and how they treat AI-written code), from the Grading queue.
To write your own, choose “Code review” under “New question”, set “Shown as” to “Pull request (diff)”, paste the diff (git diff) and fill in the title, description and CI result. Line numbers for planted problems count every line as shown, file headers included. Set “Marked automatically (%)” below 100 to have a reviewer mark the rest.
In the guide: Tests → Senior reviews of AI-written pull requests
Review an AI-written pull request Ready
In “Task catalogue”, choose “Review an AI-written pull request” and click “Use this”. You get a draft question with a worked example: code with six problems planted in it. Change the code and problems to suit the job, then send it for review as usual.
Full details
Candidates comment on the lines with problems, give each a severity, and choose Approve or Request changes. They never see the planted problems. A comment within one line of a problem counts as finding it.
It’s marked automatically: half for problems found (recall), 30% for comments that were real (precision), 10% for severity and 10% for the verdict. On the candidate’s page, the row shows recall and precision; open it to see each problem found or missed and each false alarm.
To plant problems in your own code review question, use “Planted problems” in the editor, one per line: line number | security, logic, performance or tests | low, medium or high | what’s wrong.
In the guide: Tests → Review an AI-written pull request
Letting candidates use documents, and assistant answers that are wrong on purpose Ready
In a test’s settings, “Documents candidates may use” takes one https address per line. Candidates see them as links during the test. Use sections to set steps like Debug, Build, Optimise and Explain, with points per question.
Full details
With the AI assistant on, tick “Test how carefully candidates check the assistant”. About one answer in four then has a deliberate mistake. Candidates are told before and during the test, and the conversation in your report shows which answers had one. Using the assistant is never marked down in itself.
In the guide: Tests → Letting candidates use documents, and assistant answers that are wrong on purpose
3. The question library
Questions, skills and role profiles used to build tests.
AI draft questions in the library Partly ready
The SetBenchmark library includes a large bank of questions written by AI, covering C, C++, .NET, Java, WordPress, Flutter, Laravel, React Native, React, frontend, Android, iOS, Sanity, Contentstack, DevOps, project management, QA and account management.
Full details
Each has an “AI draft” badge until a Kombee engineer has read it and its answer and approved it. You can read and copy an AI draft at any time, but you can’t put it into a test until it is approved: the question picker in a test shows reviewed questions only, and a test with an unreviewed AI draft can’t be published. Filter by skill in the library to find them.
Tests that already had AI drafts in them before reviews were required keep working. On the test page those questions show “AI draft, not reviewed”, with a note asking you to check them before inviting more candidates, or to duplicate the test and swap them.
What isn’t finished
Waiting for: Kombee review. 2,204 AI-written library questions are waiting for a Kombee engineer to read and approve them. Until approved they can be read and copied but not put into tests.
How to switch it on:
- Kombee names 2–3 engineers as library reviewers and gives them Kombee staff access (PLATFORM_ADMIN_EMAILS on the server).
- Each reviewer opens Console → Library review, reads each question and its answer, and clicks Approve or Retire.
- At about 2 minutes a question, the full library is about 75 hours of review: start with the skills Kombee hires for most.
In the guide: The question library → AI draft questions in the library
Finding questions Ready
Click “Question library”. You see SetBenchmark’s ready-made questions and your company’s own.
Full details
Search by words at the top, and filter on the left by type, AI mode, difficulty, status and source, then click “Search”. Questions reported as leaked online are hidden unless you ask for them.
On a multiple-choice, coding, SQL or fill-in-the-blank question, click “Check if AI can solve it”. An AI answers it, seeing only what a candidate sees, and is marked like a candidate. If it gets full marks the question shows “AI can solve it”, and the library can hide or show only such questions.
Tip: “AI mode” says what a question was written for: answering without AI, with AI allowed, or working with AI.
In the guide: The question library → Finding questions
Writing a question Ready
Click “New question” and choose the type: multiple choice, coding, SQL, written answer, a small web page, fill in the blanks, a diagram, a code review, a repository task, a file to upload, a data science notebook, or a prompt-engineering question.
Full details and steps
Fill in the title, the question, the difficulty, what it is written for, and the skills it tests. Coding questions also need test cases and a reference answer.
To check a coding question, paste an answer under “Mark this answer” on its page. For each test that fails you see its input, the expected output, what the answer printed and any error, hidden tests included (only authors see these; candidates never do). Each test also says what happened and how long it took: Passed, Wrong output, Didn’t compile (syntax error), Crashed while running, Too slow (time limit), Used too much memory, or Stopped by the sandbox. The same words appear on a candidate’s page, in the candidate’s own example runs and in Practice. Code is written in the same editor candidates use; open “Test against my own input” to run it on any input you type.
How to do it:
- Click “New question” and pick the type.
- Fill in the form and click “Save”.
- Click “Send for review”. A second person must approve it before it can go into a test.
Tip: A reviewer can “Retire” a question so it can’t go into new tests, and “Bring back” a retired one: it is live again with its approved version (or a draft if it was never approved).
In the guide: The question library → Writing a question
Repository tasks Ready
A repository task gives the candidate a small codebase to fix or extend, in JavaScript (also ServiceNow), TypeScript, Python, Go, Java, SAP ABAP, Salesforce, Workday integrations (XSLT) or DevOps and cloud. Choose “Repository task” under “New question”.
Full details and steps
1 · Source repository: paste a public GitHub address with a branch or tag and click “Import”, upload a .zip (up to 4 MB; for a private repository, download it as a .zip), or click “Start from the example”. Only text files are kept: dependencies, build output, .git and pictures are left out, and “files left out” lists them. At most 200 files and 1 MB in all. The question keeps its own copy, so later changes to the repository don’t affect it.
Imported code (from GitHub or a .zip) is removed 30 days after the last invitation to any test using it closes: the box “Remove this code 30 days after the last test using it closes” is ticked to start with. The files are replaced by a short note and the question is retired; scores and marks are kept. Code still in a draft test, or never used in a test, stays. If the code is yours to reuse, untick the box. To use a removed task again, import the code again.
2 · What candidates can see and change: give each file a rule. “Can change”: the candidate edits it. “Read only”: shown but locked. “Grading only”: never shown to the candidate, and added back when the answer is marked. When there are grading-only test files, only they decide the score, so tests a candidate writes can’t raise it. Tick “Candidates may add files” to let them add a test of their own.
3 · How it’s scored: the grading tests’ share (60% to start with) is split by how many grading tests pass; the rest is a rubric a reviewer marks. At 100% no rubric is needed and it’s marked automatically. Set how long one test run may take (5 to 60 seconds). For Go, a run currently takes about 40 seconds, so set 60.
4 · Solution: add the changed files that solve the task. It’s never shown to candidates. “Check the grading tests” runs the tests twice: with the solution every grading test should pass, and with the files untouched at least one should fail. The same check runs when you send the question for review.
How to do it:
- Click “New question”, then “Write a repository task question”.
- Import the files or start from the example, and pick the language.
- Set each file’s rule, the tests’ share and the rubric.
- Add the solution and click “Check the grading tests”.
- Save, then send it for review.
In the guide: The question library → Repository tasks
Agent tasks: directing a coding agent Ready
In the task editor, tick “Candidates direct a coding agent”. The test must allow AI. The candidate tells the agent what to do, and it proposes a step at a time. Nothing changes until they approve the step, and they can reject it or give a new instruction. The chat assistant is off on these tasks, so every AI step is on the record.
Full details
The agent sees only the files the candidate can see, never the grading tests or the solution. It can take 40 steps on a task. It can’t run anything: the candidate runs the tests. If your test sets the assistant to make mistakes on purpose, about one step in four is a tempting shortcut, and the candidate is told this can happen.
To see one working, the SetBenchmark library has a ready agent task: “Fix the refund total with the coding agent” (JavaScript, easy). Put it in a test that allows AI, invite yourself, and try it.
In the guide: The question library → Agent tasks: directing a coding agent
Data science notebooks Ready
A notebook question gives candidates one or more datasets (CSV, JSON or other text files, up to 500 KB each) and Python cells to work in, with pandas, NumPy, SciPy and scikit-learn and no internet. They run their cells and see each cell’s output under it, as in Jupyter.
Full details
In the editor: add the datasets (they appear read-only under data/), write the starting cells (start each with a line “# %%”), and write the grading checks as a Python unittest file that imports the notebook as notebook and tests its variables, for example self.assertEqual(notebook.total, 30). Sample checks are optional: candidates can run those. Add the solution cells, then click “Check the grading checks”: every check must pass with the solution and at least one must fail with the starting cells.
Choose how much the checks mark (70% by default); a reviewer marks the rest with the rubric, such as clear, idiomatic pandas. The Task catalogue has a ready-made one, “Data science notebook”.
In the guide: The question library → Data science notebooks
Prompt-engineering questions Ready
A prompt question asks the candidate to write a prompt for the AI. You add test cases: an input (a customer message, a document, some data) and, in plain words, what a good answer does (for example “Answers with the single word ‘billing’”). Tick “Hidden” on cases candidates shouldn’t see: they count only when the answer is marked.
Full details
When it is marked, the prompt runs on every case, with the case’s input where the prompt says {{input}} (or after the prompt), and an AI judge checks each answer against what you expected. The share of cases passed earns the cases’ part of the points (70% by default); a reviewer marks the rest with the rubric, such as a clear, robust prompt. The reviewer sees each case’s answer and verdict.
Each case uses your company’s AI allowance, both when candidates try their prompt and when it is marked. The Task catalogue has a ready-made one, “Prompt engineering”.
In the guide: The question library → Prompt-engineering questions
Role profiles Ready
A role profile lists the skills a job needs, how important each is, and the level (1 to 5) needed. Templates, the skill map and “roles that fit” all use these.
Full details
If a skill isn’t tested by any published question, the profile warns you.
In the guide: The question library → Role profiles
Skills Ready
The list of skills, with how many questions test each one. Add your own skills here.
In the guide: The question library → Skills
A client’s own codebase: cleaning it up first Ready
To test candidates on the client’s real work, import the client’s code into a repository task (a .zip from their repository), then use “Clean up a client’s code” before anything else. Under “Names to replace”, list the client’s name, products and internal systems with what to put instead, one per line, for example Acme Pay => Shop Co. They’re replaced in the code and in file names, keeping their case and style (AcmePay becomes ShopCo, acme-pay becomes shop-co), so the code still fits together.
Full details
Ticked to start with, it also replaces email addresses, internal host names and URLs (public sites such as github.com are kept), IP addresses, phone numbers (+91 and +971, or 10-digit mobiles in quotes), AWS account numbers and copyright lines. “Every change” lists what changed, file by file. Run the tests again afterwards, and keep “Remove this code 30 days after the last test using it closes” ticked.
Then, for senior hires, click “Make an AI pull request review from this code”: AI writes a pull request against the cleaned code with 6 to 8 planted problems, saved as a new draft code review question for you to check. Optionally say what it should change, for example “retries for payments”.
Tip: Secrets (keys, tokens, passwords in code) still stop the import: remove them in the client’s repository first. The clean-up runs on SetBenchmark’s server and nothing is sent to an AI; only the PR review draft uses AI, and your code isn’t used to train models.
In the guide: The question library → A client’s own codebase: cleaning it up first
Uploading code: secret scan and AI task ideas Ready
When you import a repository for a repository task, we check it for secrets such as AWS keys, private keys, tokens and passwords in code. If any are found the import stops and tells you the file and line. Remove them and import again. Real-looking email addresses are listed for you to check, but don’t stop the import.
Full details
Click “Suggest 3 tasks with AI” for a bug fix, a feature and a refactor based on the code, with how to test each. You choose and write the task. Your code isn’t used to train models.
In the guide: The question library → Uploading code: secret scan and AI task ideas
5. Marking, integrity and appeals
Answers that need a person, notes from monitoring, and candidates asking for a second look.
Marking written answers Ready
Click “Grading queue”. Written answers, web pages, diagrams and files are marked by a person. Candidates’ names are hidden so marks are fair.
Full details and steps
The numbers at the top show how many wait, how many are yours, how many are past the 48-hour target, and how well markers agree.
The AI may suggest a mark with its reasons. It is advice only: your mark counts.
How to do it:
- Click a row to open it (the oldest is already open). Each row shows the question, its type, whether the AI has a suggestion, how long it has waited and who marks it.
- Read the answer and the marking guide (rubric).
- Type the score and, if you like, a note.
- Click “Save mark”.
Tip: About one answer in ten is also given to a second marker, who marks it without seeing the first mark. This only measures how well markers agree; it never changes the candidate’s result.
Tip: Repository tasks: the grading tests have already given their share. You see how many passed, their points and output, each changed file as a diff (added lines green, removed lines struck through), and the candidate’s explanation. Enter only the rubric part: the screen adds the tests’ points and shows the total that is saved.
In the guide: Marking, integrity and appeals → Marking written answers
Working with AI: scoring how a candidate used the assistant Ready
This is the AI fluency review. When a question allows the built-in AI assistant and the candidate used it, the question waits for a “working with AI” score once they submit. Click “Grading queue”, then “Working with AI” at the top. Candidates’ names are hidden here too.
Full details
Open a question to see: how many prompts they sent, how much of their own code came from the AI (after their edits), how many suggested changes they dismissed, and whether they ran the tests after applying one. The timeline shows every prompt, reply and action, with the time from the start of the test, and the final answer.
With AI in your plan, click “Ask the AI for a pre-read”. The AI reads the timeline and suggests a score for each part of the rubric, with its reasons, and the phases of the work. It only suggests: the scores start filled in with its suggestions and you change any of them.
Score each part from 1 (weak) to 4 (strong): breaks the problem down, clear and scoped prompts, checks AI output, judgement (rejects bad ideas), and owns the final result. Add a comment for the hiring team and click “Submit score”. The score is out of 20, kept apart from the answer’s own score, and shown on the candidate’s page.
Tip: You can change a score later: open the question again from the candidate’s page.
In the guide: Marking, integrity and appeals → Working with AI: scoring how a candidate used the assistant
AI Fluency score Ready
On a test that allows AI, each candidate who used the assistant gets an AI Fluency rating, 0 to 100, on their page under the test. It has four parts: Prompt effectiveness (clear questions that lead to usable suggestions, without dozens of retries), Critique and changes (editing or turning down suggestions instead of pasting everything), Verification (running the tests after taking AI code) and Result (how much passes the tests).
Full details
Bands: 75 and up is Strong, 50 to 74 Developing, under 50 Needs support. Until someone marks the “working with AI” rubric, the number is worked out from what the candidate did in the editor, and says so. Once a reviewer marks it, their marks replace the estimate, and the result is added to the candidate’s skill profile as the skill “AI Fluency”, and to their Talent Passport when they add that test to it. The estimate alone never goes on either.
Guarded and unguarded questions: on a question where AI is allowed (unguarded), copying and pasting is how people work with the assistant, so those signals don’t count towards the integrity risk there. They still appear on the integrity review, marked “AI allowed on this question: not counted”. On every other question (guarded) they count as usual.
In the guide: Marking, integrity and appeals → AI Fluency score
Agent sessions: how a candidate supervised the agent Ready
Click “Grading queue”, then “Working with AI”. Agent tasks are listed under the questions. Click “Review” to see the supervision log: what the candidate asked, what the agent proposed, what they decided and how quickly, with the grading tests, files changed and what the agent cost. “Terminal” lists every command they ran, with the time, the exit code and the end of the output.
Full details
Score five things from 1 to 4: kept the agent on the real problem, rejected unsafe or lazy fixes, reviewed changes before approving, kept scope tight, and whether the result is correct and tested. The AI can suggest numbers, but yours are the ones recorded. This is kept apart from the answer’s own mark.
In the guide: Marking, integrity and appeals → Agent sessions: how a candidate supervised the agent
Click “Find developers”. You see developers who opted in, matched on verified skills, with no names. You can only filter on skills, location, notice and pay: never age, gender, school or years of experience. Your company can ask for 20 introductions a month, each with a short note. If the developer accepts, they appear in your candidates with their name and email. They can decline without giving a reason.
Full details
On a test or demo site you may see sample profiles: their headline starts with “Sample profile” and their reference with “dev-SAMPLE”. They are there so you can try searching and asking for an introduction. Nobody receives those introductions.
In the guide: Marking, integrity and appeals → Find developers: job matching with the community
For the company that runs the community. Click “Developer community” to offer a published test as a practice problem, contest or exam, and to see passed exams waiting for a certificate. Check the attempt and its integrity notes, then click “Issue certificate”. When a contest ends, check for copying, then click “Publish standings”. You can revoke a certificate with a reason, and its public check then says so.
In the guide: Marking, integrity and appeals → Running the developer community
Integrity review Ready
Click “Integrity review” to see where cheating may have happened. Every monitored test gets a risk score from 0 to 100, built from time away, shortcut keys, pastes, camera checks and similar code. Low is under 40, Medium is 40 up to High, and High starts at 70 unless you change it.
Full details
Only High opens a case. To open cases for Medium too, click “When to review” and tick “Open a case for Medium risk too”. A Medium test with no case still shows its Medium badge on its own page.
The “How often reviewers agree” table shows, per kind of signal, how many flags people confirmed and dismissed. Once a kind has 20 labels, the confidence shown on cases becomes that share, so dismissing false alarms recalibrates it. If people mostly dismiss a kind, it isn’t telling you much.
In the guide: Marking, integrity and appeals → Integrity review
Reviewing one test Ready
See each signal with its time, the question it happened on, the rule behind it, its confidence, how many risk points it adds (a kind your reviewers mostly dismiss counts one step lower, and says so), and pictures if the test took them. Flags the candidate disputes in an appeal are marked “Disputed by the candidate”. Dismiss or confirm each one with a reason, and ask the candidate to explain if you want their side.
Full details
Click “Summarise with AI” for a short AI summary of the integrity case, a suggested action and, when there is code, an estimate of how likely AI wrote it. This is advice for you: it never decides anything.
Tip: What the checks notice is never proof on its own. Always look at the whole picture.
Tip: Assistive technology and shared office networks can look unusual. Exemptions you agree with a candidate switch the related checks off.
In the guide: Marking, integrity and appeals → Reviewing one test
Nothing is chosen for you Ready
On a candidate’s page, and on an integrity review, no option is selected and no reason is typed. Pick the decision yourself and write the reason: every decision keeps one, of at least 10 characters. For integrity, the wording shown in the box is only an example. “Advance together” on the candidate list needs a reason too, kept with each decision.
In the guide: Marking, integrity and appeals → Nothing is chosen for you
Blocked until reviewed Ready
If an attempt still needs a person’s decision, the candidate can’t be advanced, alone or with others, and their overall score and skill scores are hidden on their page and in Compare until the case is decided. Rejecting or putting them on hold still works.
Full details
The candidate page, Compare and the Integrity pages all show the same 0 to 100 integrity risk.
In the guide: Marking, integrity and appeals → Blocked until reviewed
Advancing several candidates together Ready
Tick the candidates, write a reason and click “Advance together”, then confirm. The reason is kept with each decision. Anyone with no test result, an open integrity case or an identity hold is refused, and you are told who.
In the guide: Marking, integrity and appeals → Advancing several candidates together
Scores that aren’t final yet Ready
While written answers wait for a marker, the candidate’s page says “Provisional” and how many points are marked so far. Skills that depend on unmarked answers show low until then.
In the guide: Marking, integrity and appeals → Scores that aren’t final yet
Too little evidence for a skill Ready
A skill with no answer given isn’t assessed, so a blank test says nothing about it. A skill resting on fewer than 10 points shows “Early sign only”.
In the guide: Marking, integrity and appeals → Too little evidence for a skill
Problems candidates report Ready
When a candidate uses “Report a problem” during a test, it appears under “Needs your attention” on Home with its reference, until someone clicks “Mark handled” beside it on the candidate’s page. Dropped connections are noted on the attempt and don’t appear there.
In the guide: Marking, integrity and appeals → Problems candidates report
The same email twice Ready
Adding a candidate whose email is already in use is refused and tells you who has it. To add them anyway, for example for another role, tick “Add even if this email is already a candidate”.
In the guide: Marking, integrity and appeals → The same email twice
A name that doesn’t match Ready
If the name typed at the start shares no word with the name invited, it becomes a signal on the attempt, adds to identity risk and sends the attempt to Integrity for a person to look at. A shorter name or a different order isn’t flagged.
In the guide: Marking, integrity and appeals → A name that doesn’t match
Integrity modes: Standard and Strict Ready
Every test has an integrity mode, shown at the top of “Monitoring” on the test’s details. Pick one and the checks below are set for you. Change any check and the mode shows as Custom.
Full details and steps
“No monitoring”: no camera, screen or focus checks. Every test still checks answers for copied code and for how they were written (below).
“Standard”: the camera presence check (one person in view, every 2 minutes) and focus tracking (time away, shortcut keys, pastes).
“Strict”: everything in Standard, plus the identity check (ID and selfie, then face matching), a photo at the start, sharing the whole screen, one monitor only, pasting from outside the test turned off, “Explain your code” on coding questions, and an AI style check on longer answers (these two use AI credit).
Candidates are told exactly what their test checks before they start, and agree to it.
How to do it:
- Open the test and click to edit its details.
- Under Monitoring, choose No monitoring, Standard or Strict.
- Change any single check if you need to (the mode then shows as Custom), and save.
Tip: With pasting turned off, a candidate can still copy and paste within their own answer, and questions that allow AI are left alone. A stopped paste is noted with its length, never the text.
In the guide: Marking, integrity and appeals → Integrity modes: Standard and Strict
Explain your code Ready
With “Explain your code” on (it is part of Strict), each coding question has a box under the code: “Explain your code”. The candidate writes a few sentences on how it works and why they chose that approach. Questions that allow AI don’t ask.
Full details and steps
After the test is marked, an AI checks whether the explanation describes this code: its approach, data structures and edge cases. Spelling, grammar and language don’t count; a short, plain, accurate explanation scores high. The result shows under “AI-likelihood per answer” on the integrity review page, with what didn’t match. An explanation that doesn’t match (under 40 out of 100) is flagged as “The explanation didn’t match the code” for a person to look at; one that matches well lowers the answer’s AI-likelihood.
How to do it:
- Open the test’s details and, under Monitoring, choose Strict, or tick “Explain your code”.
- After candidates finish, open their integrity review and read “AI-likelihood per answer”.
Tip: A mismatch is a reason to ask the candidate about their code, never proof on its own.
In the guide: Marking, integrity and appeals → Explain your code
Software cameras, remote desktops and automation Ready
On monitored tests the browser also notes three things that can mean someone else is doing the test. A software camera (OBS, ManyCam, Snap Camera and the like) can play a recording or someone else’s face: “A software camera was used”. A graphics driver that remote-desktop sessions use: “Looks like a remote-desktop session”. A browser reporting that automation software is driving it: “The browser was driven by automation software”.
Full details
Each shows on the integrity timeline with the rest and counts towards the risk score. They are best effort: many remote-access tools can’t be seen from a web page, so none of them is proof, and their absence doesn’t prove anything either.
In the guide: Marking, integrity and appeals → Software cameras, remote desktops and automation
AI-likelihood per answer Ready
A minute or two after a test is marked, every code, SQL and written answer gets an AI-likelihood: Unlikely, Possible or Likely, with a score out of 100 and the reasons in plain words. It is on the integrity review page, under “AI-likelihood per answer”.
Full details
It comes from how the answer was written, read from its autosaved drafts (the same ones the replay shows): code that arrived between two saves faster than anyone types, with no paste recorded (“Code arrived all at once”); a long answer typed from start to end with almost no corrections, as when copying from another screen (“Typed straight through”); and pastes. On monitored tests the typing rhythm counts too: how fast and how evenly keys were pressed (never which keys). Keys pressed at a perfectly even, machine-like rhythm (“Typed at an even, machine-like rhythm”) suggest a typing tool, and far more code than keys pressed, with no paste (“More code appeared than keys were pressed”), suggests code put in some other way. On Strict tests an AI also reads longer answers and says whether they read like AI-written work; on its own that is weak evidence, so it moves the score only a little.
What it finds also appears as signals on the timeline, counts towards the risk score like any other signal, and can be dismissed by a reviewer. On questions that allow AI it is shown but never counts. Answers too short to judge say “Too short to tell”.
Tip: It is an estimate to tell you where to look, never proof. Click “Replay how it was written” to watch the answer being typed before deciding.
In the guide: Marking, integrity and appeals → AI-likelihood per answer
If a browser may have been tampered with Ready
Monitoring runs in the candidate’s browser, so the server checks it too. If a monitored test finishes without ever reporting that monitoring was on, the attempt gets a signal. If the page reports full screen but the window doesn’t fill the screen, that is a signal too. These are for a person to look at, not proof: a zoomed browser can cause the second one.
In the guide: Marking, integrity and appeals → If a browser may have been tampered with
The evidence behind a similar-code flag Ready
Under “Code similarity”, each match shows two scores. “Beyond common code” leaves out code that many of the question’s other answers share (the usual way to solve it), so it only measures what the two answers share beyond that; this is the score that flags. “All code” counts everything. Answers that are near-copies of each other don’t make their code “common”, so a group of copies, or an answer leaked to many candidates, still flags.
Full details
Click “See the evidence” for the two answers side by side: shared lines are highlighted, and lines many answers share are grey, with how many share them. “When the shared code appeared” says, for each candidate, when it showed up in their autosaves, how many minutes into the test, whether it arrived in one save or over several, and whether a paste came just before it. Code that arrived in one save just after a large paste reads differently from code written over many saves.
“How this is measured” explains the method in plain words. The check can’t tell who wrote the code first, and similar code is a reason to look, never proof on its own.
In the guide: Marking, integrity and appeals → The evidence behind a similar-code flag
Replaying from a signal, and signals that count less Ready
On an integrity review, a signal tied to a question has a “Replay from here” link. It opens the answer playback at the moment the signal happened and starts playing from there, so you see the answer as it was then and what the candidate typed next. Click “Pause” to stop on a save.
Full details
When people have reviewed at least 20 of one kind of signal and confirmed fewer than 30%, that kind counts one step less towards risk: high counts as medium, medium as low. “How often reviewers agree” on the Integrity page marks these “Counts less towards risk”. This only ever lowers risk, and it updates as reviews come in.
In the guide: Marking, integrity and appeals → Replaying from a signal, and signals that count less
Deciding an integrity case: Clear, Re-test, Invalidate, or Reject & block Ready
Until then, that test’s scores stay hidden everywhere: on the page, in skill levels and in the printable report. They come back once a person records an outcome.
Full details
Under “Decision”, choose one. Clear: no misconduct, the result stands. Re-test: this attempt stops counting and the candidate gets a free retest with different questions, which never uses a credit. Invalidate attempt: this attempt stops counting, and the candidate stays in the process. Reject & block: the application is rejected and that email address can’t be invited again.
Write the reason the candidate reads. It’s required, and it must be plain: mentioning scores, thresholds or how the checks work is refused. Anything but Clear also needs a reason for your team, which the candidate never sees.
The candidate is emailed the reason and how to appeal. A Clear is only emailed to someone you had asked to explain.
In the guide: Marking, integrity and appeals → Deciding an integrity case: Clear, Re-test, Invalidate, or Reject & block
Second reviewer for Reject & block Ready
Reject & block needs two different people. The first choice shows “Pending second review” in the queue and on the case. Until a second reviewer agrees, nothing changes and the candidate isn’t told.
Full details
A different reviewer opens the case and clicks “Agree: reject and block” or “Disagree”. Disagreeing puts the case back to open. Both people need to be able to make hiring decisions. A company can switch this rule off in Settings → Policies (“Reject & block needs two people”); it is on unless switched off, and we recommend keeping it.
In the guide: Marking, integrity and appeals → Second reviewer for Reject & block
Appeals Ready
Candidates can ask for a second look at a result or an integrity decision, once per test, and can attach up to 2 files as evidence. Before asking, they see what the checks noticed in their own test (never another candidate’s name or code) and can tick the flags they dispute; the appeal lists them, and they’re marked on the integrity review. Each appeal goes automatically to someone who wasn’t part of the original call, who is emailed and has 72 hours. If 72 hours pass, they and your admins are emailed, and it shows “Past the 72 hours”. If nobody else can take it, your admins are told.
Full details
Click “Appeals”, read the reason and open any files, then answer. The candidate sees your answer. Choosing “We changed it” on an integrity appeal undoes the decision: the test counts again, a rejection is reversed, the block is lifted, and a re-test nobody started is taken back. It stays in the audit trail.
In the guide: Marking, integrity and appeals → Appeals
Identity Trail: following a candidate by browser and network Ready
The trail also shows when a candidate joined a live interview or started an AI interview, with the same browser and network details. On tests and interviews with an identity check, it also shows each face and voice score (see “Identity check”). Open a candidate and click “Identity Trail”. Each row is a moment we saw them, such as starting and handing in a test, with the browser, the network (the first three blocks of the address only), the city the network is in (approximate, from DB-IP’s free location data) and the time zone. Add notes here for checks a person made, like a short video call.
Full details
The risk card adds up what stands out: a change of browser, a network or time zone change, the same browser used by another candidate, the same email under another name, a vendor on your watch-list, each with its points. Low is under 30, Medium 30 to 69, High 70 and up. A network change alone stays Low, because offices and phone hotspots do that.
Tip: Without an identity check, faces, voices and IDs aren’t compared, so the trail can’t prove who someone is, and a Passport says so to anyone checking it.
In the guide: Marking, integrity and appeals → Identity Trail: following a candidate by browser and network
Lips and voice out of step (dubbed or deepfake video) Ready
A check against deepfakes in video interviews. During a live interview, the candidate’s own browser compares how their mouth moves on camera with the sound from their microphone. If they are more than 400 ms apart for 10 seconds, the video is probably dubbed or generated (someone else speaking, or a face-swap). The Identity Trail then shows “Lips and voice out of step” with how far apart and for how long, and it adds to the identity risk score (Medium on its own; with another sign it can reach High and hold the candidate).
Full details
It is a reason to look, never a rejection: a slow connection or an old computer can put sound and picture slightly apart, but rarely by this much for this long. Ask for a short re-check on camera if in doubt. No picture or sound is sent for this, only the numbers, and it is checked once per interview.
In the guide: Marking, integrity and appeals → Lips and voice out of step (dubbed or deepfake video)
Identity check: ID, live selfie and voice Partly ready
Turn it on per test: open the test, and under Monitoring tick “Camera presence check”, then “Identity check”. Before the test the candidate agrees to it, photographs their ID (AI reads the document type, name, date of birth and the last 4 characters of its number; the full number is never kept), takes a selfie while turning their head as asked, and can record 10 seconds of their voice. The test can’t start until this is done.
Full details
The selfie is compared with the ID photo and given a score from 0 to 1. At 0.80 or above (your match line) the candidate is verified. Between 0.70 and the line, or when the name on the ID differs from the invitation, they can carry on and a person should look. Below 0.70, or when no live person is seen turning their head (a photo held up to the camera can’t turn), they try again; after two misses they can ask a person to check them (see below). The candidate is told which of the two went wrong.
During the test, the camera takes a photo every 1 to 2 minutes, compared with the selfie. A photo that doesn’t match becomes a “face didn’t match” signal on the attempt, with the photo, and adds to the identity risk. The same face on another candidate’s identity check (another name or email) is flagged too.
Open the candidate’s Identity Trail to see the “Identity check” card: the ID details, the scores, whether a live person turned their head, and the voice sample. People who review identity (recruiters and admins) also see the ID and selfie side by side, and the photo kept with any checkpoint that didn’t match. Every photo view is logged.
Tip: Scores never reject anyone: they add to the identity risk, and a person decides. Faces and voices are compared on SetBenchmark’s own servers by open models; templates are encrypted and never used to train AI. A printed photo or a screen can’t turn its head, but the check isn’t a certified anti-spoofing system.
What isn’t finished
Waiting for: Outside approval. ID photo, live selfie, face and voice matching work. Checking an Aadhaar or other government ID against the government’s own records (DigiLocker) needs SetBenchmark to be registered as a DigiLocker partner.
How to switch it on:
- Kombee applies as a requester on partners.digilocker.gov.in (company documents, use case, data protection details).
- Once approved, share the client ID and secret with the developers to add “Verify with DigiLocker” to the ID step.
In the guide: Marking, integrity and appeals → Identity check: ID, live selfie and voice
Assisted identity checks (two selfie tries didn’t pass) Ready
When two selfie tries don’t pass, they can ask a person to check them. Either the face didn’t match the ID (old ID photos, glasses and poor light cause this), or no live person was seen turning their head (a photo held up to the camera, a poor camera, or not turning far enough). Recruiters and admins are emailed, and the candidate appears under “Assisted identity checks” on the Identity holds page, where the “Why” column says which of the two it was, and on their Identity Trail.
Full details
Look at the ID and selfie side by side, or call the candidate and ask them to show their ID. Then choose “It’s the same person: confirm” or “Not confirmed: close the check” and say how you checked. Confirming lets them start the test, and their selfie becomes what later steps are compared with. “Not confirmed” closes the check, so they can’t start the test, and puts them at High risk: they go on hold until someone chooses Clear or Reject as proxy on their Identity Trail.
In the guide: Marking, integrity and appeals → Assisted identity checks (two selfie tries didn’t pass)
First-day photo (verified to Day 1) Ready
When a candidate who had an identity check joins, open their Identity Trail and click “Make a first-day photo link” (tick “Email it to the candidate” to send it). The link works once, for 14 days, on a phone or laptop with a camera, without signing in.
Full details
On their first day they take a selfie turning their head as asked. A match marks the trail “Verified to Day 1” and tells your systems through the identity.day1.verified webhook. A photo that doesn’t match puts them at High risk, on hold, for a person to check in person.
In the guide: Marking, integrity and appeals → First-day photo (verified to Day 1)
Identity holds: when a candidate is held at High risk Ready
At High risk a candidate goes on hold automatically: your admins are emailed, they appear under “Identity holds” in the menu, and they can’t be advanced. Nobody is ever rejected automatically.
Full details
Open their Identity Trail and choose Clear, Check again (they stay on hold) or Reject as proxy. A reason of at least 20 characters is always needed, and it’s logged. Rejecting also revokes their Passport.
Tip: Turn automatic holds off under Settings, then “Identity & Passport”.
In the guide: Marking, integrity and appeals → Identity holds: when a candidate is held at High risk
Vendors: who submitted a candidate, and their scorecard Ready
The rates come from your own team’s decisions and are indicators to look into with the agency, not findings against it. Click “Vendors” to see each staffing partner’s candidates, how many reached High risk, and their proxy and integrity rates. Admins click “Add vendor”. Set who submitted a candidate on the candidate’s Identity Trail.
Full details
A vendor with at least 5 candidates at your watch line (10% unless you change it) shows “Over your line”. Click “Add to list” to watch-list them: their candidates gain 20 risk points.
In the guide: Marking, integrity and appeals → Vendors: who submitted a candidate, and their scorecard
Verified by SetBenchmark badge: sending a checked candidate to a client Ready
For staffing vendors (Kombee’s own staff augmentation team first). When a candidate has passed the identity check, or a person confirmed them, open their Identity Trail. Under “Verified by SetBenchmark badge”, type the client you’re sending them to, choose how long it lasts (30, 90 or 180 days) and whether to include test results, then click “Make the badge and ask the candidate”.
Full details
The candidate is emailed a link where they see exactly what the client will see, and agree or say no. Until they agree, the badge page shows nothing about them. The card shows each badge’s state: waiting for the candidate, valid, declined, expired, withdrawn, or not valid.
Copy the “Link for the client” into your submission, or paste the badge (the HTML under it) into an email or CV: it shows “SetBenchmark ✓ ID + live face verified”, or “not valid”, and links to the page. The client needs no account. The page shows the candidate’s name and selfie, the type of ID (never its number), when and how they were checked, how many face checks during tests and interviews matched, test results if you included them, and a signed credential they can check. You see how many times it was opened.
On the first day, the client clicks “Make a first-day link” on the badge page and opens it on the joiner’s phone or laptop: a selfie turning the head, compared with the identity check. The result shows on the badge and on your Identity Trail. Up to 3 links per badge.
Tip: A badge stops working by itself when the candidate is put on identity hold, rejected as a proxy, or withdraws their consent. The client only ever sees “not valid”, never why. To stop one yourself, click “Withdraw” and give a reason (kept for your team).
Tip: The badge says the identity was checked. It isn’t a background, criminal-record or employment-history check.
In the guide: Marking, integrity and appeals → Verified by SetBenchmark badge: sending a checked candidate to a client
Question Shield: a different version of a question for each candidate Ready
Question Shield stops answers being passed around. Open a question in the library and click “Variants”, or click “Question Shield” in the menu.
Full details
Under “Swap words”, type a word in the question and what to swap it for, separated by commas (for example Asha → Ravi, Meena). Every combination becomes a variant and goes live at once, so only swap things that leave the answer the same.
Under “Draft with AI”, ask for up to 5 drafts with new names, values and data. A draft with a different structure from the original is refused. The rest wait as “To approve”: read each one, check its answers, then click “Approve”.
Each new candidate gets the least-used live variant they haven’t seen, so a re-test gets different questions. Marking and reports work as normal, and reports count a variant under its original question.
Tip: Once at least two variants have 200 attempts each, one that runs clearly easier or harder than the others is taken out of use, and a healthy one is never taken out because another lacks attempts. The last live variant is never retired.
In the guide: Marking, integrity and appeals → Question Shield: a different version of a question for each candidate
A different version of every question for each candidate Ready
In a hiring test’s details, tick “A different version of every question for each candidate”. Once the test is published, versions are made in the background as you invite people: about one for each invited candidate, at least 3 and at most 25 per question. Each candidate gets a version they haven’t had, the least used.
Full details
AI writes versions in which the answer changes. A version goes live by itself only when a check proves that the answers to the original and to every other version are wrong for it: for multiple choice the correct option’s text is different, for fill in the blanks the accepted answers are, for coding the other versions’ reference solutions fail its tests (and its own passes), and for SQL the other queries give a different result. So a leaked answer from one candidate is useless to the next. Versions that can’t be proved this way, and every version of a written, design or code review question, wait for you under Question Shield → “To approve”. Repository tasks and file uploads keep the original.
The test page shows “A different version for each candidate”: live versions and those waiting, per question, and “Make versions now” if you don’t want to wait.
Tip: It uses your AI plan: a few requests per question as candidates are invited.
In the guide: Marking, integrity and appeals → A different version of every question for each candidate
Leak monitor: when a question is found online Partly ready
If you find a question online, click “Question Shield”, paste the address under “Check a page” and click “Compare”. If the page needs a sign-in, paste its text too. We compare it with your questions and show the match.
Full details
At 85% or more the variant that leaked is retired at once, new invitations get another one, and your admins are emailed. Below 85% it’s listed “To review” for you to retire or dismiss. Tests already taken aren’t changed.
Under “Actions”, “Removal request” gives you wording to send the site. Nothing is sent for you: send it yourself, then click “Mark as sent”.
Tip: Automatic daily searching of public GitHub code needs a search key set up by Kombee. The page says whether it’s on.
What isn’t finished
Built; reports can be added by hand. The automatic GitHub search waits for a token.
Waiting for: Token (free) (GitHub token for the leak monitor). The leak monitor is built, but has no GitHub token on the server, so it doesn’t search GitHub for leaked questions yet. Searching the wider web would also need a paid search API key.
How to switch it on once the GitHub token for the leak monitor account is bought:
- Kombee creates a GitHub account or uses its organisation’s bot account, then Settings → Developer settings → Personal access tokens → Fine-grained token, “Public repositories (read-only)”, 1 year. Copy it.
- From the repository on your own computer: scripts/set-secret.sh LEAK_GITHUB_TOKEN, paste the token. Then on the server: sudo systemctl restart setbenchmark-api setbenchmark-api2.
- In SetBenchmark, open Questions → Leak monitor and check a scan runs (it runs daily; the first results show the next day).
In the guide: Marking, integrity and appeals → Leak monitor: when a question is found online
9. Settings (admins)
Setting up your company. Click “Settings” in the left menu; each subject is in the Settings menu on the left of the page, grouped into Organisation, Account and Connect. Most of these are for company admins.
Team Ready
Invite colleagues with their email and a role; they get an email to join. Change someone’s role in the list, or click “Remove access”.
Full details
To invite many at once, open “Invite a list of colleagues”: paste one person a line (name, email) or choose a spreadsheet saved as CSV, pick one role for all of them, and click invite. Up to 50 at a time; people already on the team are skipped.
Below: make roles of your own (“Custom roles”), group people into teams, and hand a person’s tests over to someone else when they leave.
In the guide: Settings (admins) → Team
Sign-in Ready
Tick the ways people may sign in: Microsoft, Google, or email (with password, codes or passkeys). Your sign-in page shows only these. Microsoft or Google is listed only once Kombee has switched it on for SetBenchmark; until then it isn’t offered anywhere. Switching a way off also signs out, at once, anyone who signed in that way (you stay signed in).
Full details
Choose whether staff are signed out after a time without use. Candidates are never signed out during a test.
To use your company’s own single sign-on (Okta, Microsoft Entra ID, Google Workspace and others), click “Set up single sign-on” and your IT team completes it.
In the guide: Settings (admins) → Sign-in
Directory sync (SCIM): staff from your identity provider Ready
Under Settings → Sign-in, “Directory sync (SCIM)” lets your identity provider (Okta, Microsoft Entra ID, Google, JumpCloud) add, update and remove staff for you. Click “Turn on and make a token”, copy the token (it is shown once) and give it, with the SCIM base URL shown there, to your IT team for the provider’s SCIM app. “Make a new token” replaces the old one at once; “Turn off” stops the directory at once and leaves people as they are.
Full details
People the directory adds can sign in through your single sign-on. Someone it deactivates or removes is disabled and signed out at once; nobody is deleted, so their history stays. The directory can’t deactivate or demote your last active admin.
Roles come from groups: the strongest role among a person’s groups wins, and someone in no mapped group gets the default role you choose. Groups called Admins, Recruiters, Hiring managers, Interviewers, HR managers or Employees work as they are; map your own group names one a line, for example “Talent Acquisition = recruiter”. Saving the mapping gives everyone the directory manages their role again straight away.
In the guide: Settings (admins) → Directory sync (SCIM): staff from your identity provider
Appearance Ready
Your logo and colours, for staff and candidate screens. See the change before saving.
Full details
The same page sets how your emails look to candidates: the sender name, the reply-to address, and “Keep a copy of candidate emails”. With an address there, every email to a candidate is also sent, hidden, to that address for your records.
In the guide: Settings (admins) → Appearance
Messages Ready
Change the wording of emails SetBenchmark sends, such as the candidate invitation. You can always go back to the original wording.
Full details
Every email carries your company’s name, logo and main colour from Appearance, with a button for its link. Write in plain text: a line that ends with {{link}} becomes the button. Click “Preview email” to see it with made-up details before you save.
In the guide: Settings (admins) → Messages
Web address Ready
Use an address on your own domain, such as tests.yourcompany.com, instead of ours. Add it here and follow the steps to prove the domain is yours.
Full details
With your DNS provider, add the TXT record shown (it proves the domain is yours) and point the address at ours with the CNAME record shown. Then click “Check”. When the record is found, the address goes live at once: candidates open their tests there, and the links in their emails use it. The secure certificate is set up by itself on the first visit.
Tip: DNS changes can take up to an hour to show. If “Check” says the record isn’t found yet, wait a little and check again.
Tip: Removing the address stops it working at once; links go back to the SetBenchmark address.
In the guide: Settings (admins) → Web address
Hiring policies Ready
“Equality survey for the fairness report” (off at first) offers candidates a short, optional survey after a hiring test, for Analytics → Fairness. Answers are only counted in groups of 30 or more and never shown for one person.
Full details
“Blind interviews” hides who candidates are from people with the interviewer role: they see a reference like “Candidate 4F2A”, never a name, email or resume, and can’t search by name. Candidates join interviews by voice only. Recruiters and hiring managers still see everything.
“Anonymous reports” makes printed reports and result emails use that reference instead of the name, so they’re judged on the work.
Defaults for the whole company: monitoring for new tests, how sensitive the integrity signals are, whether candidates may explain integrity signals, AI marking suggestions, reminders to candidates who haven’t started (up to 3, a day, 2 days or 3 days apart, with the last the day before the test closes), receipts when they submit, whether candidates may give themselves 48 more hours once (if they haven’t started, from 2 days before the test closes up to a week after), and the extra time offered first when an adjustment is approved.
You can also lock settings for every test. “Monitoring is the same for every test” locks monitoring, so authors can’t change it. Watermark, hidden question titles and hidden points can each be off or on by default (authors choose), or always or never (fixed for every test). You can write a declaration for new tests and, if you like, make every test use it.
Fixed settings change draft tests straight away, and every new or copied test. Published tests keep their settings, because candidates may already have agreed to them. On a test, a fixed setting is greyed out with the note “Set for every test in Settings → Hiring policies”.
Under “Always on” you see the protections that can’t be switched off: no automatic rejection, a person makes every decision, adjustments are available, names are hidden while marking, and your data is never used to train AI models.
“What reviewers have taught the signals” shows how your reviewers’ decisions change the risk score. When reviewers dismiss most flags of one kind (under 30% confirmed, from 20 reviews), that kind counts one step less, but never below its floor: strong signals such as more than one face, a face mismatch, developer tools or a large paste never count less than Medium, and every other kind never counts for nothing. It goes back to normal if reviewers start confirming it again. The table lists each kind with its confirmed and dismissed counts, its weight now and its floor; “Changes” lists every time a kind started or stopped counting less.
In the guide: Settings (admins) → Hiring policies
Credits: pay per candidate Partly ready
Under Settings → Billing, “Credits” lets a company pay per candidate instead of (or as well as) a plan. One credit is used when a candidate starts a hiring test. Starting the same test twice uses one credit. Re-tests after an integrity review are free. Employee and bench tests are free up to 4 per person in any 12 months (fair use); a fifth in that time uses a credit when they start it, and the bench page names who that applies to when you send the invitations.
Full details and steps
Pick your currency (INR, AED, SAR, AUD, NZD or USD), then click “Buy” on a pack. You pay on the payment page and the credits arrive when the payment is confirmed. Tax is added by currency: GST 18% for INR, VAT 5% for AED, VAT 15% for SAR, GST 10% for AUD, GST 15% for NZD. Paid orders have an “Invoice” link; put your VAT or tax number (TRN in the UAE) in Billing details to show it there.
Most tests use one credit. A test with a lab (a repository task run in the sandbox, such as DevOps, SAP, Salesforce or ServiceNow) uses 2, and so does an AI interview, because they cost more to run. If the wallet has fewer than that, nothing is taken and the candidate is told the test is temporarily unavailable. A refund gives back what the test took.
Credits last 12 months and the oldest are used first. Admins get an email at 20% and 5% of the last top-up. If credits run out, a candidate who tries to start is told the test is temporarily unavailable and admins are emailed. “History” shows every change.
A company on a paid plan doesn’t need credits: its tests keep starting as before, even with credits in the wallet.
Credits suit a company that hires only a few people a year: there is no subscription and nothing to cancel, and you pay only when a candidate starts a test. On the Free plan, credits pay for tests with automatic marking and monitoring; AI help and interviews come with a paid plan. While your company pays with credits, your candidates’ code runs don’t count against the plan’s monthly limit (the credit covers their test), so nobody is stopped halfway through a coding question. Each candidate can run code up to 300 times a day.
The public Pricing page opens in the currency of the country the visitor is in: rupees in India, dirhams in the UAE, riyals in Saudi Arabia, Australian dollars in Australia and New Zealand, and US dollars elsewhere. Anyone can switch currency at the top of the page.
How to do it:
- Click “Settings”, then “Billing”.
- Under “Credits”, pick a currency.
- Click “Buy” on a pack and pay.
What isn’t finished
Works end to end in test mode.
Waiting for: Account and live keys (Razorpay live payments). Payments work end to end, but in Razorpay test mode (test keys): no real money moves. Live mode needs Kombee’s Razorpay account to finish KYC. Charging in dirhams, riyals, Australian or US dollars also needs Razorpay to switch on international payments.
How to switch it on once the Razorpay live payments account is bought:
- Kombee signs up (or signs in) at dashboard.razorpay.com with the company’s details and finishes KYC (PAN, GST, bank account). Razorpay then activates live mode.
- For AED, SAR, AUD and USD packs: in the Razorpay dashboard, ask for “International payments” to be switched on (Account & Settings → Payment methods).
- Switch the dashboard to Live mode. Go to Account & Settings → API keys → Generate key. Copy the Key ID (starts with rzp_live_) and the Key secret.
- Go to Account & Settings → Webhooks → Add new webhook. URL: https://api.setbenchmark.com/webhooks/razorpay. Choose a secret (a long random text) and tick these events: payment_link.paid, payment.failed, subscription.authenticated, subscription.activated, subscription.charged, subscription.pending, subscription.halted, subscription.cancelled. Save.
- In SetBenchmark, sign in as Kombee staff and open the console (Console → Services and AI models). Under “Payments”, choose Razorpay and paste the Key ID, Key secret and Webhook secret. Click Save, then “Test”.
- Buy the smallest credit pack with a real card, check the credits arrive in Settings → Billing and the invoice shows the right tax. Refund it from the Razorpay dashboard if wanted.
- Set the final pack prices in Console → Plans (today’s prices are samples), and ask Kombee’s accountant whether to charge UAE VAT (5%) and Saudi VAT (15%) without a registration there.
In the guide: Settings (admins) → Credits: pay per candidate
Coupons on credit packs and automatic refunds Partly ready
If you have a coupon, type it in the box next to a pack before clicking “Buy”. The percentage comes off the pack before tax, and the invoice shows the coupon.
Full details
If a credit is taken but the test never begins because of a failure on our side, the credit comes back by itself after 15 minutes, and the ledger says why. For anything else, such as a test that failed part way through, use “Give back” on the ledger.
What isn’t finished
Works in test mode.
Waiting for: Account and live keys (Razorpay live payments). Payments work end to end, but in Razorpay test mode (test keys): no real money moves. Live mode needs Kombee’s Razorpay account to finish KYC. Charging in dirhams, riyals, Australian or US dollars also needs Razorpay to switch on international payments.
How to switch it on once the Razorpay live payments account is bought:
- Kombee signs up (or signs in) at dashboard.razorpay.com with the company’s details and finishes KYC (PAN, GST, bank account). Razorpay then activates live mode.
- For AED, SAR, AUD and USD packs: in the Razorpay dashboard, ask for “International payments” to be switched on (Account & Settings → Payment methods).
- Switch the dashboard to Live mode. Go to Account & Settings → API keys → Generate key. Copy the Key ID (starts with rzp_live_) and the Key secret.
- Go to Account & Settings → Webhooks → Add new webhook. URL: https://api.setbenchmark.com/webhooks/razorpay. Choose a secret (a long random text) and tick these events: payment_link.paid, payment.failed, subscription.authenticated, subscription.activated, subscription.charged, subscription.pending, subscription.halted, subscription.cancelled. Save.
- In SetBenchmark, sign in as Kombee staff and open the console (Console → Services and AI models). Under “Payments”, choose Razorpay and paste the Key ID, Key secret and Webhook secret. Click Save, then “Test”.
- Buy the smallest credit pack with a real card, check the credits arrive in Settings → Billing and the invoice shows the right tax. Refund it from the Razorpay dashboard if wanted.
- Set the final pack prices in Console → Plans (today’s prices are samples), and ask Kombee’s accountant whether to charge UAE VAT (5%) and Saudi VAT (15%) without a registration there.
In the guide: Settings (admins) → Coupons on credit packs and automatic refunds
Languages, quiet hours and dropped connections Ready
Click “Settings”, then “Languages & connection”. Tick Arabic or Hindi and candidates get a language choice on their home page. Arabic reads right to left, and code stays left to right. These cover the main candidate screens, not your own questions, which stay as you write them.
Full details
Quiet hours stop reminders overnight in your time zone (9 pm to 8 am unless you change them). They go out when quiet hours end. A candidate gets up to 3 reminders.
Pauses give time back when a candidate’s connection drops: up to 3 per test of up to 5 minutes each, unless you change it. 0 turns them off.
Tip: All data is kept in India today. Other regions need new cloud regions, which Kombee must approve.
Tip: On a poor connection, monitoring events and snapshots wait in the browser and are sent again when the connection comes back, so evidence is delayed rather than lost. They wait only while the test page stays open.
In the guide: Settings (admins) → Languages, quiet hours and dropped connections
Candidate reports (skill reports for candidates) Ready
Click “Settings”, then “Candidate reports”. After a hiring test, each candidate gets a skill report on their assessments page, and an email when it is ready. It shows their overall level, their level in each skill with one line on how it went, their strongest skill and the one to work on, up to 4 free learning resources for skills under 80%, and when they may retake the test or ask to. It never shows the questions, their answers or your decision.
Full details and steps
“When candidates see it”: as soon as the test is marked (the default), or only after a person decides on the application. A report also waits while a person is still marking written answers, and isn’t shown while an integrity concern is recorded: the candidate sees a neutral note and a link to ask for a review.
You choose whether to show percentages (off: level names only, like “Advanced”), how they compare (“Top 25%”, once at least 5 others have taken the test), the line of feedback per skill, and the free learning. The feedback is written from scores and kinds of question only; any line sharing 8 words in a row with a question is left out.
“Days before a retake” (14 at first; 0 says nothing about retakes): if the test allows retakes, candidates start one themselves after this wait, and it uses a credit; if not, the report says they can ask you. “How long each stage should take” sets the date candidates see as “we aim to update you by” for assessment, marking, interview and decision.
Under “Learning you suggest”, add your own free learning for a skill: a title, an https address and, if you know it, the minutes it takes. Yours come first. SetBenchmark adds well-known free resources when a skill’s name matches, for example MDN for JavaScript, the Python tutorial, SQLBolt for SQL or A Tour of Go.
How to do it:
- Click “Settings”, then “Candidate reports”.
- Choose what candidates see and when, and click “Save report settings”.
- To suggest learning, choose the skill, type a title and address, and click “Add learning”.
In the guide: Settings (admins) → Candidate reports (skill reports for candidates)
Privacy requests Ready
Where your data is kept, candidates on legal hold, and candidates’ requests for a copy of their data or its deletion. The law gives 30 days; overdue requests are marked.
In the guide: Settings (admins) → Privacy requests
Data retention Ready
How long camera pictures, recordings and scores are kept. After that they are deleted automatically, except for candidates on legal hold.
In the guide: Settings (admins) → Data retention
Support access Ready
Ask SetBenchmark support a question; the answer appears here. You also choose whether, and for how long, support may look at your account (read-only). Every visit is listed.
Full details
Tip: The status page (/status) shows whether everything is working and any planned maintenance.
In the guide: Settings (admins) → Support access
Integrations Partly ready
Connect your HR system, calendars and other tools. For webhooks, open their activity to see what was sent to each address in the last 30 days, and switch an address on or off.
Full details
Calendar invites for interviews: click “Connect” on Google Workspace Calendar or Microsoft 365 Calendar and sign in with the account interviews should come from (for example a shared hiring account). After that, scheduling an interview sends each interviewer and the candidate a calendar invitation from that account, and moving or cancelling the interview updates it. “Test” checks the connection still works.
Tip: Without a connected calendar, people still get the interview email with a calendar file to add it themselves.
What isn’t finished
Webhooks, API keys and HR-system sync work. Calendar connection waits for the OAuth apps.
Waiting for: OAuth apps (free, need Kombee’s accounts) (Google and Microsoft calendar connection). Companies can’t yet connect their Google or Microsoft calendar, because SetBenchmark’s own OAuth apps aren’t registered. Until then interview invitations go by email with a calendar file.
How to switch it on once the Google and Microsoft calendar connection account is bought:
- Google: in Google Cloud console (project setbenchmark) → APIs & Services → enable “Google Calendar API”. OAuth consent screen: External, app name SetBenchmark, scopes calendar.events. Credentials → Create OAuth client ID → Web application.
- Authorised redirect URI (Google) and Redirect URI (Microsoft): https://api.setbenchmark.com/integrations/oauth/callback
- Microsoft: in Microsoft Entra admin centre → App registrations → New registration, “Accounts in any organisational directory and personal Microsoft accounts”, add the redirect URI above, API permissions → Microsoft Graph → Calendars.ReadWrite (delegated), then Certificates & secrets → New client secret.
- On the server, add the client IDs to /etc/setbenchmark.env (GOOGLE_OAUTH_CLIENT_ID=…, MICROSOFT_OAUTH_CLIENT_ID=…) and the secrets with scripts/set-secret.sh GOOGLE_OAUTH_CLIENT_SECRET and scripts/set-secret.sh MICROSOFT_OAUTH_CLIENT_SECRET (from the repository, on your own computer).
- Remove “calendar” and “oauth” from MOCK_PROVIDERS in /etc/setbenchmark.env, then restart: sudo systemctl restart setbenchmark-api setbenchmark-api2.
- Submit the Google app for verification (Google asks for it before outside companies can connect). In SetBenchmark, Settings → Integrations → Connect Google Calendar, and schedule a test interview.
In the guide: Settings (admins) → Integrations
Connecting Sqalible or another HR system Partly ready
So that Quality of Hire and the Bench stay up to date without spreadsheets, your HR system sends SetBenchmark who was hired, how they’re doing, and who is on the bench. Click “Settings”, then “Integrations”, and find “Sqalible, or another HR system”.
Full details
1. Under “API keys”, make a key named after the system (for example “Sqalible”) with only “Send hires, how they performed, and the bench”. That key can do nothing else. 2. Give it to the HR system’s team with the two calls shown on the card: hires and outcomes (employee number, email, role, joining date, rating and rating period, probation, leaving date and reason), and the bench (people between projects, and who has left the bench for a project). Nightly is plenty. 3. The card lists every call that arrives: when, from which key, how many rows, what they did, or why one was refused.
Each hire is matched by employee number and joining date and linked to their SetBenchmark candidate by email, so use the same email the candidate applied with. Quality of Hire then compares their test scores with their ratings, probation and whether they stayed; people sent on the bench can be reassessed with a free bench campaign, matched to client demand, and given learning with a re-test.
Tip: HR managers and admins can see the card. Until the system is connected, import CSV files on Quality of Hire and on the Bench.
What isn’t finished
Waiting for: Outside API details. Sqalible (or any HR system) can already send hires, outcomes and the bench to SetBenchmark with its own key. Pulling from Sqalible needs its API documentation and a login, which aren’t public.
How to switch it on:
- Kombee asks Sqalible for its API documentation and a test account.
- Share them with the developers; the pull job then runs on a timer next to the existing push.
- Meanwhile: Settings → Integrations → “Sqalible, or another HR system” shows the address and key Sqalible can push to today.
In the guide: Settings (admins) → Connecting Sqalible or another HR system
API keys Ready
Keys for your own systems to use SetBenchmark’s API. A sandbox key works in a separate, empty test company made just for it. It can’t see or change your real data, uses no credits, and never emails or messages anyone. Anything it adds stays in the test company and doesn’t appear on your screens.
Full details
The key is shown once, when it is made. Revoking a key asks you to confirm, and takes it off the screen.
In the guide: Settings (admins) → API keys
Usage Ready
What your company used this month (AI, code runs, tests, interview minutes, emails) against your plan, the last six months, and which features it came from.
Full details
Set a monthly spending cap if you like: AI work pauses at the cap, and tests already under way still finish.
Admins get an email when the company reaches 50%, 80% and 100% of a monthly limit or of the spending cap, once each a month. If one jump crosses more than one mark, you get one email for the highest.
In the guide: Settings (admins) → Usage
Billing Partly ready
Your plan, every plan you can switch to, your invoices and the details printed on them. Each plan card shows its price, what it includes and how much of each thing you get a month (for example 100 tests taken, then ₹60 for each extra one).
Full details
To change plan, a company admin clicks “Choose” on a plan card. A free plan starts straight away. For a paid plan you go to Razorpay’s own page to pay by card, UPI or net banking. The plan starts as soon as Razorpay confirms the payment; until then you keep what you have, and the page says it is waiting for your payment, with a “Pay on Razorpay” button. Have a coupon? Open “Have a coupon?” on the card before choosing.
Moving from one paid plan to another keeps the card on file: the new plan starts at once and only the difference for the rest of the month is charged. Moving down to the free plan, or “Cancel plan”, keeps your paid plan working until the end of the month you paid for; then you move to the free plan. Changed your mind? “Keep my plan”.
Tip: Only company admins can change the plan. Others see the plans and the current one.
Tip: A payment that fails on renewal shows “Payment failed”: Razorpay tries again, so check the card with your bank.
Tip: Left the Razorpay page without paying? Use “Pay on Razorpay” again, choose another plan, or “Don’t pay now”. Nothing is charged until you pay.
What isn’t finished
Plans, invoices and usage work; card payments are in test mode.
Waiting for: Account and live keys (Razorpay live payments). Payments work end to end, but in Razorpay test mode (test keys): no real money moves. Live mode needs Kombee’s Razorpay account to finish KYC. Charging in dirhams, riyals, Australian or US dollars also needs Razorpay to switch on international payments.
How to switch it on once the Razorpay live payments account is bought:
- Kombee signs up (or signs in) at dashboard.razorpay.com with the company’s details and finishes KYC (PAN, GST, bank account). Razorpay then activates live mode.
- For AED, SAR, AUD and USD packs: in the Razorpay dashboard, ask for “International payments” to be switched on (Account & Settings → Payment methods).
- Switch the dashboard to Live mode. Go to Account & Settings → API keys → Generate key. Copy the Key ID (starts with rzp_live_) and the Key secret.
- Go to Account & Settings → Webhooks → Add new webhook. URL: https://api.setbenchmark.com/webhooks/razorpay. Choose a secret (a long random text) and tick these events: payment_link.paid, payment.failed, subscription.authenticated, subscription.activated, subscription.charged, subscription.pending, subscription.halted, subscription.cancelled. Save.
- In SetBenchmark, sign in as Kombee staff and open the console (Console → Services and AI models). Under “Payments”, choose Razorpay and paste the Key ID, Key secret and Webhook secret. Click Save, then “Test”.
- Buy the smallest credit pack with a real card, check the credits arrive in Settings → Billing and the invoice shows the right tax. Refund it from the Razorpay dashboard if wanted.
- Set the final pack prices in Console → Plans (today’s prices are samples), and ask Kombee’s accountant whether to charge UAE VAT (5%) and Saudi VAT (15%) without a registration there.
In the guide: Settings (admins) → Billing
Audit log Ready
Who did what, and when. For checking and for audits.
In the guide: Settings (admins) → Audit log
AI use Ready
Every kind of task the AI has done for your company, with the model used. Below, a log of the latest AI calls: click “Details” to see exactly what was sent and returned.
Full details
Tip: The AI only suggests. People make every decision.
In the guide: Settings (admins) → AI use
AI in SetBenchmark: what it is for, its limits, and who decides Ready
SetBenchmark uses AI (Claude models from Anthropic) to help people, never to make the decision. It drafts questions, versions of a question and tests; suggests marks with reasons for answers that can’t be marked automatically; runs AI interviews that say they are an AI at the start; suggests a next step for a candidate with the evidence; sums up the signals behind an integrity flag; reads ID card photos and compares faces and voices; and does smaller jobs such as estimating a question’s difficulty.
Full details
What it is not for: rejecting, ranking out or failing anyone (SetBenchmark refuses a rejection no person made), and judging emotion, personality, confidence, honesty, accent, appearance, age, gender, ethnicity, religion, caste, nationality or disability. Every scoring prompt forbids those, and a scoring prompt without that rule is refused. Candidate names are not sent to the AI when it marks answers.
Its limits: marking suggestions can be wrong, above all for open answers and unusual but valid approaches. Speech-to-text makes more mistakes with some accents and noisy rooms, so candidates see and correct the words before sending a spoken answer, and the recording is kept. Signs of AI help (pastes, time away, similarity) are not proof: they come with their evidence and can be dismissed or appealed. Face and voice matching can fail with poor light, a poor camera, illness or a changed appearance; a failed match never rejects anyone. AI-written library questions can contain errors, so they can’t go into a test until a SetBenchmark engineer has reviewed them.
Oversight: every suggestion shows its reasons and is accepted, changed or ignored by a person. Every AI call is logged in full (what was sent and returned, the model and its version, the prompt version, the cost and the time), and each score links to its log entry under Settings → AI use. Fairness figures are under Analytics. Candidates can appeal a flag and see its evidence. The AI providers used don’t train on this data, and a candidate’s AI logs are erased with their other data. Before launch, the same answers written in Indian, Australian and Gulf English are marked by the AI and must score alike.
Tip: This is SetBenchmark’s note on intended use, limits and human oversight, written to the NIST AI Risk Management Framework and ISO/IEC 42001. Share it with candidates or auditors who ask.
In the guide: Settings (admins) → AI in SetBenchmark: what it is for, its limits, and who decides
Slack alerts Ready
Click “Settings”, then “Integrations”, and find “Slack”. In Slack, open Apps → Incoming Webhooks → Add to Slack, choose the channel and copy the address (it starts https://hooks.slack.com/services/). Paste it, tick the alerts you want and click “Connect”.
Full details
Alerts: a candidate finishes a test, a test is fully marked, a decision is recorded, identity risk is high. Each message says what happened and links to the candidate in SetBenchmark; no names or scores are sent to Slack. “Test connection” posts a hello to the channel. Connect again to add another channel. The address works like a password, so it is never shown again.
In the guide: Settings (admins) → Slack alerts
Results in Greenhouse or Lever Partly ready
Click “Settings”, then “Integrations”, and scroll to “Results in your ATS”. Paste the API key and click “Connect”. Greenhouse also needs the id of the user notes are written as.
Full details
When a result is complete, the score, each skill and a report link are added as a note on the candidate, once. Integrity status is left out unless you tick “Include integrity status” on that connection. This works for candidates who came from that ATS, with its id as their external reference. If a send fails, the card shows the problem.
Tip: This hasn’t been tried with a live Greenhouse or Lever account yet.
What isn’t finished
Waiting for: Partner test account. Results are sent to Greenhouse and Lever through their published APIs, but the connection hasn’t been tried against a real Greenhouse or Lever account.
How to switch it on:
- Kombee (or a client using Greenhouse or Lever) provides a sandbox account or a Harvest/Lever API key.
- In Settings → Integrations, connect it, send one candidate’s result, and check it appears on the candidate in Greenhouse or Lever.
In the guide: Settings (admins) → Results in Greenhouse or Lever
10. What candidates see
The candidate’s side, from the invitation email to the end of the test. Candidates need no account.
Their assessments Ready
The link in the invitation email signs the candidate in and shows their tests and interviews. Each link works once. If it no longer works (opened in another browser, say), they type their email under “Email me a new link” and a new one is sent, but only to the address the company has for them, and only while a test is still open.
Full details
“Your application” shows the steps that apply to them (application received, assessment, marking, interview, decision), which are done, and for the current one “we aim to update you by” a date the company sets. It says when a decision is made, never what it is.
If the company allows it, a test they haven’t started shows “Give me 48 more hours” in its last 2 days (or up to a week after it closed). It works once per test. We send more than one reminder: reminders come by email, up to 3 times, the last the day before the test closes.
In the guide: What candidates see → Their assessments
Talent Passport: saving and sharing skills Ready
On a skill report, the candidate clicks “Save to Passport”. Only fully marked results with no open integrity or identity question can go on. Each skill lasts 12 months from the test.
Full details
From “Your Talent Passport” on their home page they make a share link lasting 7, 30 or 90 days, choosing whether it shows scores and percentile. The link and QR code show once. They see every view, and can stop a link at any time.
Whoever opens the link sees whether the signature checks out, the skills and when each runs out. An expired link says “Link expired” and shows nothing else. The Passport is a signed credential that anyone can check with our public key.
Tip: A company can revoke a Passport from the Identity Trail, and an Invalidate or Reject & block decision revokes it too. Companies can put their own brand on Passports, or turn them off, under Settings, then “Identity & Passport”.
In the guide: What candidates see → Talent Passport: saving and sharing skills
Sharing your verified identity badge with a client Ready
When a staffing company sends you to a client, they may ask to include a “Verified by SetBenchmark” badge. You get an email; the link shows exactly what the client will see: your name and the selfie from your identity check, the type of ID you showed (never its number), when you were checked, how many face checks matched, and your test results if included.
Full details
Click “I agree, share it” or “No, don’t share it”. Nothing is shown before you agree. On your first day the client may ask for a quick selfie turning your head; it’s compared with your identity check.
Tip: The badge stops if the staffing company withdraws it, when it expires, or if you withdraw your identity-check consent on your “Your data” page.
In the guide: What candidates see → Sharing your verified identity badge with a client
If the connection drops during a test Ready
Candidates can keep working. Every change is kept in their browser straight away and sent as soon as they’re back, so a drop or a closed tab loses nothing. If the page reloads, unsent answers come back by themselves.
Full details
After a drop of 30 seconds or more, the time lost is given back, up to your limits, and never more than has passed since the last answer reached us. Each pause shows on the test’s integrity timeline.
In the guide: What candidates see → If the connection drops during a test
Language and WhatsApp reminders for candidates Partly ready
With more than one language on, candidates choose theirs at the top of their home page. It then applies across their screens: consent, device checks, their details, the test, help, Your data, the Passport, interviews and AI interviews. Questions and company messages stay as the company wrote them. Arabic and Hindi need a native speaker’s review before launch.
Full details
Under “Reminders on WhatsApp” they can add their number and agree to reminders. We never message anyone on WhatsApp who hasn’t said yes, and messages hold no scores. If WhatsApp can’t send, it falls back to email. On a candidate’s Timeline tab, “Messages sent” shows each message and whether it was sent, delivered or read.
Tip: WhatsApp needs a Meta Business account set up by Kombee in the console. Until then reminders go by email.
What isn’t finished
Languages work. WhatsApp and SMS wait for a messaging account.
Waiting for: Account, sender registration and templates (SMS and WhatsApp messages). Reminders by SMS and WhatsApp are built (MSG91, Gupshup or Meta’s WhatsApp Cloud API), but no messaging account is connected, so only email goes out.
How to switch it on once the SMS and WhatsApp messages account is bought:
- SMS in India: Kombee registers on a DLT portal (as TRAI requires), registers a sender ID and the message templates, then opens an MSG91 account and copies its Auth key, the DLT sender ID and the template ID.
- WhatsApp: Kombee verifies its business in Meta Business Manager, adds a WhatsApp number, and gets the reminder template approved. Either use Meta’s WhatsApp Cloud API directly (copy the permanent access token, Phone number ID and template name) or Gupshup (copy API key, app name, number and template ID).
- For Meta: in the Meta app, set the WhatsApp webhook to https://api.setbenchmark.com/webhooks/whatsapp so replies and opt-outs come back.
- In SetBenchmark, sign in as Kombee staff and open the console (Console → Services and AI models). Under “SMS and WhatsApp”, choose the provider and paste its fields. Save, then “Test” with your own number.
- Invite yourself to a test with SMS/WhatsApp ticked and check the message arrives in your language.
In the guide: What candidates see → Language and WhatsApp reminders for candidates
Withdrawing consent to being captured Ready
Before a monitored test, the candidate sees where their data is stored and which law protects it. After the test they can open “Your data” and click “Withdraw my consent”. Their camera images, screen captures and recordings are deleted straight away, and they get a deletion certificate with a code. Their answers and results stay; to delete those too, they make a deletion request.
Full details
This can’t be done during a test, or while a legal hold is on. A full deletion also leaves a certificate, which holds only counts and a one-way reference, not the person’s details. The wording for each country needs confirming by local counsel before launch.
In the guide: What candidates see → Withdrawing consent to being captured
The step before the questions Ready
On a monitored test, the candidate turns on the camera, screen sharing and full screen after clicking “Start the test”. The time this takes is given back, up to 5 minutes in all, and is on the record. If the browser refuses full screen, they can click “Continue without full screen”, and the company is told.
In the guide: What candidates see → The step before the questions
Their skill report Ready
When a hiring test is marked (or after the company decides, if it chooses), the candidate gets an email and “See your skill report” appears next to the test. The report shows their overall level (and percentage and band, such as “Top 25%”, if the company shows them), each skill with its level and a line on how it went, their strongest skill and the one to work on, free learning for skills to improve, and when they may retake the test or ask to.
Full details
It never shows the questions, the answers or the hiring decision. While written answers are still being marked it says so; while an integrity concern is recorded it shows a neutral note and a link to ask for a review.
“Download as PDF” opens the print window to save it. At the end, one question asks how likely they are to recommend the company’s hiring process (0 to 10); the answer is anonymous and asked once.
In the guide: What candidates see → Their skill report
Retaking a test from the report Ready
If a test allows retakes, the candidate can start one from their skill report once the wait has passed (14 days unless you change it in “Settings”, “Candidate reports”). They click “Retake with a new version” and go straight to the test. Each retake uses a credit, like any other try.
Full details
If the test has no retakes left, the report says they can ask you instead. There is no button while another try is open, after you have decided on their application, or while the test is under integrity review.
In the guide: What candidates see → Retaking a test from the report
Sharing a skill report by link Ready
On their skill report, the candidate clicks “Share a read-only link” and copies the address. It is shown only once and works for 30 days. Making a new link stops the old one, and “Stop sharing” ends it straight away.
Full details
Whoever opens the link sees the candidate’s name, levels and feedback, following your settings for percentages and bands. They do not see the free learning, the retake option or anything about the hiring process. The link shows “Link expired” after it ends, and the report isn’t available while you switch reports off or while the test is under review.
In the guide: What candidates see → Sharing a skill report by link
Report language Ready
The skill report appears in the language the candidate chose: English, Arabic or Hindi. A link they share is always in English. Arabic and Hindi need a native speaker’s review before launch.
In the guide: What candidates see → Report language
Messages on a candidate’s own clock and in their language Ready
When a candidate opens their assessments page, their browser’s time zone is saved. Reminders then wait for the end of quiet hours on their clock; for anyone else, your company’s time zone is used.
Full details
The test reminder, the “report ready” message and the decision stage message go out in the language they chose (English, Arabic or Hindi), unless you wrote your own version of that message. Other emails are in English for now.
If a WhatsApp message can’t be sent, the same message goes by SMS to that number, then by email.
On a weak connection, or with the browser’s data saver on, camera images taken during a test are made smaller, about 160p.
In the guide: What candidates see → Messages on a candidate’s own clock and in their language
Telling candidates a decision is near Ready
When someone at your company records a decision, the candidate gets a message saying their application has reached the decision stage and that you will contact them. It never says what you decided. It goes by WhatsApp if they said yes to it, otherwise by email. To turn it off, click “Settings”, then “Candidate reports”, and set “Message when the decision stage is reached” to “Don’t send”.
In the guide: What candidates see → Telling candidates a decision is near
Candidates proving who they are (ID and selfie) Partly ready
On a test with an identity check, after agreeing to what the test captures, the candidate is told what the identity check collects, why, where it is kept and for how long, and agrees. Then they photograph their ID (upload a photo, or hold it up to the camera), take a selfie (look at the camera, then turn their head left and right in the order shown), and can record a sentence for the voice check.
Full details
If the selfie doesn’t match, they try once more; after that they can ask a person at the company to check them. They can withdraw their consent on their “Your data” page: their photos and face and voice templates are then deleted.
What isn’t finished
Waiting for: Outside approval. ID photo, live selfie, face and voice matching work. Checking an Aadhaar or other government ID against the government’s own records (DigiLocker) needs SetBenchmark to be registered as a DigiLocker partner.
How to switch it on:
- Kombee applies as a requester on partners.digilocker.gov.in (company documents, use case, data protection details).
- Once approved, share the client ID and secret with the developers to add “Verify with DigiLocker” to the ID step.
In the guide: What candidates see → Candidates proving who they are (ID and selfie)
Before you start Ready
Before starting, the candidate sees the sections with their time and AI rule, how the test works, and what is monitored. They agree to monitoring, fill in any details the company asks for, and pass the checks: browser, internet connection, and (for monitored tests) their full name. If the company wrote a declaration, they read it and tick “I agree to this declaration”; “Start the test” stays greyed out until they do.
Full details
Tip: “Try the editor first” opens practice questions: no timer, nothing saved.
In the guide: What candidates see → Before you start
Asking for an adjustment Ready
Candidates can ask for extra time, screen-reader support, a version without the camera, or something else. They don’t need to give a reason.
In the guide: What candidates see → Asking for an adjustment
Sections, details and their own time Ready
The candidate sees exactly what is in the test, and that their request was sent.
In the guide: What candidates see → Sections, details and their own time
The test screen Ready
The test fills the screen. The bar at the top has the AI rule, the timer, “Report a problem”, text size and Help, with a thin line showing how much is answered. On the left, numbered squares jump to a question: green is answered, amber is marked for review. Coding and SQL questions show the question beside the editor on wide screens. Answers save automatically. “Mark for review” flags a question to come back to. “Report a problem” sends a note to the hiring team with a reference. Some tests show the candidate’s email faintly across each question (a watermark). On coding questions, “Check all tests” (when the company allows it) says how many tests their saved code passes, hidden ones included; it can be used once every 15 seconds.
Full details
Tip: A– and A+ at the top change the text size on every screen.
In the guide: What candidates see → The test screen
Multiple choice Ready
Pick one or more answers. Wrong answers may lose points if the test says so before it starts.
In the guide: What candidates see → Multiple choice
Whiteboard (system design) questions Ready
A whiteboard question asks the candidate to draw a design: boxes, arrows and labels on a whiteboard that saves as they draw. Above it, “Show us” lists what the reviewer looks for (the question’s marking criteria), with a reminder that neatness doesn’t count and clear reasoning does.
Full details
Below the whiteboard, “Explain your design” is optional but recommended: the trade-offs, how it fails and recovers, what they would monitor. It shows a word count. The drawing and the explanation are one answer, and the question counts as answered once either has something in it.
Markers see the drawing with the explanation under it in the grading queue, and the AI’s marking suggestion reads both.
In the guide: What candidates see → Whiteboard (system design) questions
Repository tasks (fix the codebase) Ready
Some tasks are set on a platform such as SAP ABAP, Salesforce, ServiceNow, Workday or Terraform. They run in a sandbox that stands in for the platform (TASK.md says what it covers), with no internet and no account to sign in to. A repository task shows the project’s files on the left, the file you have open in the middle, and your changes under it. The task notes (TASK.md or README.md) open first. Files marked “ro” are read only; a changed file shows M and a new one A. If the task allows it, type a path under the files and click “Add file”, for example to add a test.
Full details
Click “Run tests” to run the project’s tests with your changes. You see how many passed and failed, the command and the output. Nothing is saved or marked when you run them, and you can run them once every 5 seconds. The tests that mark your answer may include ones you can’t see.
The “Terminal” tab runs one command at a time on your files as they are now (for example ls, cat a file, or a script of your own), in the same sandbox as the tests with no internet. Files a command writes aren’t kept; change files in the editor. Each command and the end of its output are recorded for the reviewer. If the project has a web page (an .html file), the “Preview” tab shows it with its own stylesheets and scripts as you edit; it can’t load anything from the internet.
“Your changes” lists each changed file with lines added and removed; “Undo” puts a file back as it was (or removes a file you added). Under it, “Explain your fix” is read by the reviewer with your changes. Everything saves as you type. Repository tasks need a laptop or desktop computer, and can’t be run in practice tests.
In the guide: What candidates see → Repository tasks (fix the codebase)
Seeing what the checks noticed, and asking for a review Ready
After handing in a test, a candidate can open “Ask for a review” on it. Above the form, “What the checks noticed in your test” lists each thing the checks recorded: what it was, which question, how many minutes into the test, how long or how many characters, how many risk points it counted for, and whether a person dismissed or confirmed it. For similar code it says how much of their code is like another candidate’s, beyond what many answers share; “See your matching lines” shows their own answer with those lines highlighted. The other candidate’s name and code are never shown.
Full details
In the form, they can tick the flags they dispute and explain why. Someone who wasn’t part of the original decision answers within 72 hours.
In the guide: What candidates see → Seeing what the checks noticed, and asking for a review
Reviewing code or a pull request Ready
A code review question shows code, or a pull request: its title, what the author says it does and what CI reported, then the change as one list of lines, with added lines in green, removed ones in red and each file’s name above its lines.
Full details
Click a line to comment on it. On reviews marked automatically, give each comment a severity, write a short summary for the author and choose Approve or Request changes. Comment on the line where the problem is: a comment one line away still counts.
In the guide: What candidates see → Reviewing code or a pull request
Directing the coding agent Ready
Some repository tasks have a coding agent on the right of the screen. Write an instruction, and it proposes a step. Read the step, then choose Approve, Reject or Revise instruction. Nothing changes in your files until you approve. Your instructions, its steps and your decisions are recorded, and a reviewer looks at how you supervised it, not only at whether the tests pass. Run the tests yourself: the agent can’t.
In the guide: What candidates see → Directing the coding agent
Data science notebooks (candidates) Ready
A notebook question shows the datasets at the top (open one to see its first lines) and your Python cells under them. Write code in the cells, add a cell below with “Add a cell below”, and click “Run all”: every cell runs in order and its output shows under it, including the value of a cell’s last line, as in Jupyter. If a cell raises an error, the cells after it don’t run.
Full details
Read the files from data/, for example pd.read_csv('data/orders.csv'). pandas, NumPy, SciPy and scikit-learn are there; there is no internet. If the question has sample checks, “Run the sample checks” runs them on your cells. Hidden checks mark it when you finish, and a reviewer may mark part of it too. Your cells save as you type.
In the guide: What candidates see → Data science notebooks (candidates)
Prompt questions (candidates) Ready
A prompt question asks you to write a prompt for an AI. Put {{input}} where each case’s input should go. Under it are the cases you can see, each with what a good answer does. Click “Try it on these cases”: your prompt runs on each one and you see the AI’s answer and whether it passed. You can try 20 times, once every 10 seconds.
Full details
More cases you can’t see are used when your answer is marked, so write a prompt that works for any input of that kind, not just the ones shown. Your prompt saves as you type.
In the guide: What candidates see → Prompt questions (candidates)
Competitions and hackathons Ready
Click “Competitions” to run a coding competition or hackathon on one of your own tests: a campus drive, a hiring event or an internal contest. Choose a published test, the times it opens and closes, individuals or teams (2 to 10 people a team), and prizes by place. Or click “Draft with AI”: say what it is for and which skills, and the AI writes the name, summary, rules and prize ideas and picks library questions into a new draft test. A drafted competition shows “Waiting for approval”: check every question, publish the test, then click “Approve”. Nobody can be invited before that.
Full details
Open a competition and paste the people to invite, one a line: name, email (and team). Each gets an email with their link and the times. In team mode every member takes the test; a team counts as the average of its members and is timed by its slowest. Standings update as people finish; participants see them too, with first names and initials only, marked provisional.
Copying check: pairs of entries whose code looks copied (from different people or teams; teammates may share) are listed. For each flagged person choose Clear or Disqualify and give a reason; nothing is chosen for you. A disqualified member takes their team out. Once the competition has closed, every answer is marked and every flag is decided, click “Publish results”. Every finisher gets a signed rank certificate with a badge (Winner for first, Runner-up for second and third, Top 10%, or Finisher) and any prize for their place, by email. Anyone can check one at /awards/ and its code. Participants find the competition on their start page.
In the guide: What candidates see → Competitions and hackathons
The developer community: practice, contests and certificates Ready
Anyone 18 or older can join for free at /dev/join on the community’s address, with a name, email and username. We email a sign-in link, so there’s no password. Practice problems, contests and exams are ordinary tests, and practice is separate from employer assessments: nothing goes to a company unless the developer says so.
Full details
Each problem, contest and exam can be taken once. While a contest runs, people who entered see provisional standings. Final standings appear after a person has checked them for copying (entries whose code looks copied are listed for them on the community page); then every finisher is emailed a signed rank certificate with a badge, listed under “Your rank certificates”. A person also reviews every passed exam before the certificate is issued. Anyone can check a certificate by its code at /certificates/ and the code. Under Certifications, “Add to LinkedIn” opens LinkedIn’s “Add licence or certification” form already filled in with the name, dates and the check link.
Certificates are digitally signed (a W3C verifiable credential), so the check page says whether the signature is genuine, and an employer can check it themselves with the public key at /v1/public/passports/key. Exams use only questions written for the community, never the shared library that hiring tests use, so exam answers can’t be seen anywhere else.
Renewing: a certificate lasts as long as the exam says (two years unless set otherwise). We email you 30 days before it expires. From 60 days before expiry (or any time after), the exam shows “Renew: take it again”. Pass, and once a person has reviewed it you get a new certificate with a new code; the old one says it was renewed.
Your profile is private until you turn it on. On your profile page you choose who sees your name, verified skills, practice activity and contest ranks. Only certificates count as verified skills. “Your data” lets you download everything or delete your account.
In the guide: What candidates see → The developer community: practice, contests and certificates
Job matching for developers Ready
Job matching is off until you turn it on. Companies then see your verified skills, location and notice period, but not your name. When one asks for an introduction you get an email and have 7 days. “Share my profile” gives them your name and email and makes you a candidate there. “Not now” needs no reason. Your minimum pay is private and only used to filter, and companies you hide never see you.
In the guide: What candidates see → Job matching for developers
Using the AI assistant (tests that allow AI) Ready
On a question that allows AI, the AI assistant sits under the question. The candidate is told to use it the way they would at work: reviewers look at how they plan, ask, check suggestions, test them and fix mistakes, not only at the final answer.
Full details
The note “What’s recorded” says what the company sees: the messages, the assistant’s replies, which suggested changes were applied or dismissed, and the test runs.
When the assistant suggests a change to the code, it shows “Suggested change” with how many lines it adds and removes. “Apply” puts it into the answer, “Apply and edit” does the same and brings the editor into view, and “Dismiss” leaves the answer as it is. The assistant has never seen hidden tests or answers.
In the guide: What candidates see → Using the AI assistant (tests that allow AI)
Timed sections and extra time Ready
In a test with timed sections, the top shows the section and its own timer. “Finish section” moves on; there is no going back. Some tests carry the time left over to the next section; the “Finish section” message says so.
Full details
If the company gives extra time, a green note says so: “We had a problem, not you”.
In the guide: What candidates see → Timed sections and extra time
Submitting Ready
Before submitting, the candidate sees how many questions are answered and any marked for review.
In the guide: What candidates see → Submitting
Done Ready
The candidate gets a reference number and can rate the test (anonymously). If time ran out, the page says the saved answers were submitted.
In the guide: What candidates see → Done
The optional equality survey Ready
If the company has turned it on, the page after submitting a hiring test offers a short survey: gender, age band and where the candidate lives. Every question has “No answer” and “Prefer not to say”, and “Skip” closes it.
Full details
The page tells candidates that the answers are kept apart from their result: nobody deciding on their application sees them, they never change the score, and they are only counted in groups of 30 or more to check that tests are fair.
In the guide: What candidates see → The optional equality survey
A live interview (candidates) Ready
A live interview opens from the candidate’s home page at the time it was booked (the room opens 15 minutes before). It has video and sound, a shared code editor the interviewer also types in, and chat. Nothing to install: it runs in Chrome, Edge, Firefox or Safari.
Full details
For a design question, click “Open the whiteboard”: you and the interviewer draw on the same board and see each other’s shapes within a couple of seconds. The interviewer may also open your test code in the editor and ask you to extend it.
In the guide: What candidates see → A live interview (candidates)
An AI interview Ready
Some companies ask for an AI interview: a conversation with an AI interviewer, typed or spoken. The candidate is told it is an AI, and a person at the company reads it and decides.
Full details
To answer by voice, they press “Start answering”, speak, then “Stop”. The words are written out for them to check and correct before “Send answer”. In Chrome and Edge the words appear while they speak; in other browsers, such as Firefox and Safari, they appear after “Stop” (when the company’s plan includes speech). They can always type instead.
In the guide: What candidates see → An AI interview
AI avatar: a video face for the AI interview Partly ready
When Kombee has switched on the avatar service, candidates can also choose “I agree, start with the video interviewer”. A video face says each question aloud, and they answer by speaking, as in a voice interview. The screen always says “AI interviewer, not a person”.
Full details
The avatar only says the interviewer’s own questions, word for word: it never makes anything up. The candidate’s camera stays off and only their spoken answers are recorded. If the video can’t connect within about 25 seconds, or the video interviewer’s time runs out partway through, the questions are read aloud instead and the interview carries on. If the video arrives late, it says the questions from then on. If the sound doesn’t start, they click “Turn on the interviewer’s sound”.
What isn’t finished
Live with Tavus (face “Raj Business”), on Tavus’s free plan: 20 minutes in all, a watermark, and interviews capped at 10 minutes.
Waiting for: Paid Tavus plan (Tavus video avatar). Tavus is connected and its Check passes, but the account is on the free plan (20 minutes in all, with a watermark), which is for trying it out, not for candidates.
How to switch it on once the Tavus video avatar account is bought:
- Kombee upgrades the Tavus account (platform.tavus.io → Billing) to Starter or Builder, and signs Tavus’s data processing agreement.
- In SetBenchmark, sign in as Kombee staff and open the console (Console → Services and AI models). Under “AI interviewer avatar”, open “Switch provider” (Tavus is listed first, “on now”), set “Longest interview (minutes)” within the plan’s longest conversation (Builder: 15), and click “Save settings”. The key and PAL ID stay as they are.
- Run one AI interview yourself as a candidate, choose “I agree, start with the video interviewer”, and check the face speaks each question and stops when you answer.
In the guide: What candidates see → AI avatar: a video face for the AI interview
Practice: problems to warm up on Ready
Candidates can practise before the test, and anyone can practise at /practice on the candidate site, with no account: candidates find it through “Practise first” on their home page, and developers in the community through Practice. Nothing done there is seen by employers.
Full details
Choose an area, difficulty or kind (code or multiple choice). In a coding problem, write a solution in Python, JavaScript, TypeScript, Java, C++, C# or Go, click “Run the examples” (or tick “Run on my own input”), then “Check against every test”. Examples show the expected and actual output; hidden tests show only whether they passed. Try as often as you like.
“Show a hint” opens hints one at a time. “Show the write-up” explains a full solution. Questions and tips go under Discussion; they appear once a Kombee moderator approves them. Progress is kept in the browser, so clearing its cookies starts afresh.
Tip: “You can run code 60 times an hour” means a short break is needed. If practice says it has reached its daily limit, it opens again the next day; tests are never affected.
In the guide: What candidates see → Practice: problems to warm up on
Mock interview: rehearse and get feedback Ready
From Practice, click “Practise an interview”. Type the role you are practising for and, if you like, paste a job description or the skills to be asked about. Click “Start by typing”, or “Start by speaking” in Chrome or Edge (the browser writes your words out for you to check; nothing is recorded).
Full details
The same AI interviewer companies use asks about four questions. At the end you get feedback: a short summary, what went well, what to work on, and a rating per area. No company sees practice interviews, and nothing is decided. “Stop here” ends one early, without feedback.
Up to 3 practice interviews a day for each browser; your earlier ones are listed on the same page.
In the guide: What candidates see → Mock interview: rehearse and get feedback
Their data Ready
Under “Your data” candidates can ask for a copy of their data, or for it to be deleted, and follow their requests.
In the guide: What candidates see → Their data
Help Ready
Answers to common questions, such as what to do if the connection drops.
In the guide: What candidates see → Help
11. Platform console (Kombee staff)
For Kombee staff only: every company on SetBenchmark.
Library review: approve AI-written library questions Ready
Click “Library review” in the console. It lists the AI-written library questions no person has checked yet, with how many are waiting and how many have been reviewed. Each shows its skills, level, the question and its answer (the right options are ticked; for coding and other kinds, open “Tests, answer and settings”), and how many tests already use it.
Full details and steps
Click “Approve” if the question is right and fair: companies can then put it into tests. Click “Retire” if it is wrong or not worth fixing: it leaves the library. Until a question is approved, companies can read and copy it but not use it in a test, so no AI-written question reaches a candidate without a person reading it first.
How to do it:
- Open the console and click “Library review”.
- Read the question and its answer.
- Click “Approve” or “Retire”.
In the guide: Platform console (Kombee staff) → Library review: approve AI-written library questions
Costs: daily cost report per company and per candidate Ready
Click “Costs” in the console. It shows what SetBenchmark cost Kombee to run yesterday and this month, per company, split into AI, sandbox (code runs), interviews and other; the average cost of one candidate this month for each kind of assessment (test, test with a lab, AI interview, live interview); and the recordings, spoken answers and evidence pictures stored for each company.
Full details
AI costs are exact (from each AI call’s logged cost). Sandbox and video time are estimates, from the sandbox server’s monthly price and the video service’s price per minute; the rates are at the bottom of the page.
The same summary is emailed to Kombee’s staff every morning (about 8:30 in India) for the day before.
“Fallback lines shown to candidates” counts how often an AI interview candidate got one of the four fixed, pre-approved questions instead of the AI’s own: when the safety check refused the AI’s question, gave no clear answer twice, or the AI couldn’t be reached (an outage or a spending cap). A candidate never sees a blank turn. The morning email gives the count too.
In the guide: Platform console (Kombee staff) → Costs: daily cost report per company and per candidate
Daily AI cap per company Ready
Every company has a daily cap on its AI spending, so one bad day can’t use a month’s budget. Unless Kombee sets one, the default applies (₹1,000 a day). Days start at midnight UTC (5:30 in India).
Full details
Open the company in the console. “Daily AI cap” shows today’s AI spending against the cap. Type a new amount in rupees and click “Save daily cap”, or leave it empty and save to go back to the default.
At 80% and at 100% the company’s admins and Kombee’s staff get an email. At the cap, new AI work (marking suggestions, question writing, reports) waits until the next day. An AI interview or test assistant a candidate is already using carries on, up to half as much again, so nobody is cut off mid-answer; if even that runs out, the candidate gets a fixed question instead of a blank turn.
In the guide: Platform console (Kombee staff) → Daily AI cap per company
Growth: where new companies come from Ready
Click “Growth” in the console. It shows the companies created in the last 90 days and how far they got: made a test, published one, invited a candidate, got a first result, and the typical time from signing up to the first test. These numbers come from SetBenchmark’s own records, so they count every company, whether or not its people accepted cookies.
Full details and steps
“Self-serve sign-ups by source” shows where companies that signed up on the website came from: the campaign tags on the link they first arrived by (utm_source, utm_medium, utm_campaign), else the website that sent them, else “direct”. This is known only when the visitor allowed analytics cookies, or arrived on the sign-up page with the tags. “Campaigns” and “Sign-ups per week” break it down further.
At the bottom, “Google Tag Manager for the public website” holds the container ID (like GTM-ABC1234) that Kombee’s marketing team set up with Google Analytics inside it. Paste it and click Save: the website picks it up within a minute, no deploy. Leave it empty and save to switch website analytics off.
How to do it:
- Create a Tag Manager web container and a GA4 property, and add the GA4 tags in the container (docs/analytics/plan.md lists the events).
- Publish the container in Tag Manager.
- Open the console, click “Growth”, paste the container ID and click Save.
Tip: Tag every link you share with utm_source, utm_medium and utm_campaign (lower-case, for example utm_source=linkedin&utm_medium=cpc&utm_campaign=q1-hiring) so sign-ups show under the right campaign.
Tip: Google Analytics only sees visitors who accept cookies, so its numbers are lower than the console’s.
In the guide: Platform console (Kombee staff) → Growth: where new companies come from
Cookies on the public website Ready
SetBenchmark’s own public pages (home, pricing, sign-up, sign-in and the legal pages) ask visitors about cookies the first time they come. “Accept all”, “Reject all” and “Choose” are side by side; the website works the same whichever you pick. Nothing from Google loads until you agree.
Full details
Analytics cookies count visits and pages (Google Analytics) and remember for 30 days which campaign or website brought you, so Kombee can see which ones lead to sign-ups. Marketing cookies show whether an ad led to a visit. Neither holds your name or email.
To change your mind, click “Cookie settings” at the bottom of any of these pages. Withdrawing removes the cookies. Company sites, the candidate site, tests, interviews and the console never use analytics or marketing cookies.
In the guide: Platform console (Kombee staff) → Cookies on the public website
Credit packs and giving credits Partly ready
Open “Plans” to change credit pack prices in each currency (before tax), or take a pack off sale. Packs show on the public pricing page at /pricing and in companies’ Settings → Billing. The prices there now are samples until the price list is approved.
Full details
Open a company and use “Give credits” for a deal or pilot: the number of credits and a reason. A company not on a paid plan then needs credits to start tests.
For credit payments to arrive, tick the “payment_link.paid” event on the webhook in the Razorpay dashboard.
What isn’t finished
Giving credits works. Paid packs are in test mode, and prices are samples.
Waiting for: Account and live keys (Razorpay live payments). Payments work end to end, but in Razorpay test mode (test keys): no real money moves. Live mode needs Kombee’s Razorpay account to finish KYC. Charging in dirhams, riyals, Australian or US dollars also needs Razorpay to switch on international payments.
How to switch it on once the Razorpay live payments account is bought:
- Kombee signs up (or signs in) at dashboard.razorpay.com with the company’s details and finishes KYC (PAN, GST, bank account). Razorpay then activates live mode.
- For AED, SAR, AUD and USD packs: in the Razorpay dashboard, ask for “International payments” to be switched on (Account & Settings → Payment methods).
- Switch the dashboard to Live mode. Go to Account & Settings → API keys → Generate key. Copy the Key ID (starts with rzp_live_) and the Key secret.
- Go to Account & Settings → Webhooks → Add new webhook. URL: https://api.setbenchmark.com/webhooks/razorpay. Choose a secret (a long random text) and tick these events: payment_link.paid, payment.failed, subscription.authenticated, subscription.activated, subscription.charged, subscription.pending, subscription.halted, subscription.cancelled. Save.
- In SetBenchmark, sign in as Kombee staff and open the console (Console → Services and AI models). Under “Payments”, choose Razorpay and paste the Key ID, Key secret and Webhook secret. Click Save, then “Test”.
- Buy the smallest credit pack with a real card, check the credits arrive in Settings → Billing and the invoice shows the right tax. Refund it from the Razorpay dashboard if wanted.
- Set the final pack prices in Console → Plans (today’s prices are samples), and ask Kombee’s accountant whether to charge UAE VAT (5%) and Saudi VAT (15%) without a registration there.
In the guide: Platform console (Kombee staff) → Credit packs and giving credits
Revenue by region and plan Ready
Click “Revenue” in the console. It shows credits sold, given, used, given back and expired over the last 12 months, then credit pack and plan revenue by region, before tax. Each currency stays separate, so nothing is added across currencies.
Full details
Once a day SetBenchmark also checks orders still waiting for payment against Razorpay. A paid order whose confirmation never arrived gets its credits, and any wallet that doesn’t match its ledger is written to the server log.
In the guide: Platform console (Kombee staff) → Revenue by region and plan
Sandbox use Ready
Click “Sandbox use” in the console to see code runs per company over the last 30 days. Each run gets its own sandbox, which is destroyed when the run ends, so nothing is left running. A few sandboxes are started ahead of time for tests being taken or opening within half an hour, so a run starts at once; each is still used for one run only. Cost shows only what was recorded per run; the sandbox server itself is a fixed monthly cost.
In the guide: Platform console (Kombee staff) → Sandbox use
Plan prices in other currencies Partly ready
In the console, open “Plans”, then “What companies see” on a plan. Under “Price on the public page, in other currencies”, type the price in AED, SAR, AUD, NZD and USD and click “Save”. A currency left empty shows “Ask us” on the pricing page. Prices are placeholders until Kombee approves the price list.
Full details
The pricing page opens in the visitor’s likely currency, worked out from their country or browser language, and they can always switch.
What isn’t finished
Prices show in each currency; paying in AED, SAR, AUD or USD needs international payments.
Waiting for: Account and live keys (Razorpay live payments). Payments work end to end, but in Razorpay test mode (test keys): no real money moves. Live mode needs Kombee’s Razorpay account to finish KYC. Charging in dirhams, riyals, Australian or US dollars also needs Razorpay to switch on international payments.
How to switch it on once the Razorpay live payments account is bought:
- Kombee signs up (or signs in) at dashboard.razorpay.com with the company’s details and finishes KYC (PAN, GST, bank account). Razorpay then activates live mode.
- For AED, SAR, AUD and USD packs: in the Razorpay dashboard, ask for “International payments” to be switched on (Account & Settings → Payment methods).
- Switch the dashboard to Live mode. Go to Account & Settings → API keys → Generate key. Copy the Key ID (starts with rzp_live_) and the Key secret.
- Go to Account & Settings → Webhooks → Add new webhook. URL: https://api.setbenchmark.com/webhooks/razorpay. Choose a secret (a long random text) and tick these events: payment_link.paid, payment.failed, subscription.authenticated, subscription.activated, subscription.charged, subscription.pending, subscription.halted, subscription.cancelled. Save.
- In SetBenchmark, sign in as Kombee staff and open the console (Console → Services and AI models). Under “Payments”, choose Razorpay and paste the Key ID, Key secret and Webhook secret. Click Save, then “Test”.
- Buy the smallest credit pack with a real card, check the credits arrive in Settings → Billing and the invoice shows the right tax. Refund it from the Razorpay dashboard if wanted.
- Set the final pack prices in Console → Plans (today’s prices are samples), and ask Kombee’s accountant whether to charge UAE VAT (5%) and Saudi VAT (15%) without a registration there.
In the guide: Platform console (Kombee staff) → Plan prices in other currencies
Companies Ready
Add a company, see its plan and usage, suspend it, export or remove its data. Support access works only while the company allows it.
Full details
The Subdomain is the company’s own web address, such as acme-labs. It fills in from the company name, and you can change it before saving. A few names are kept for SetBenchmark’s own addresses (such as api, take, www, custom and console), so they can’t be used. The admin gets an invitation email with a sign-in link as soon as you add the company.
When you add a company, tick the modules it may use (all are ticked to start with). Its addresses work straight away, with nothing to set up on the server. On a company’s page, “Modules” switches modules on or off at once; putting it on a plan, or changing its plan, sets them from the plan again.
In the guide: Platform console (Kombee staff) → Companies
Plans Ready
Plans, prices, coupons and each company’s subscription and invoices. Everything here is up to Kombee: no plan, price or limit is fixed in the software.
Full details
“Add the suggested plans” adds four to start from: Free (₹0: tests and basic monitoring, 10 tests a month), Silver (₹4,999 a month: adds video interviews and AI help with questions), Gold (₹14,999: adds AI interviews, employee skills and SMS or WhatsApp) and Platinum (₹39,999: everything, including API access and single sign-on). Prices are before GST. Change any of them, or make your own.
Each plan has “What companies see”: its name, description, place in the list and whether companies are offered it. A plan not offered can still be given to a company from its page in the console, for example for a special deal. Prices, modules and monthly allowances change by publishing a new version; companies already on the plan keep their version until moved.
A price of 0 makes a free plan. It never goes to Razorpay. When a paid plan or the free trial ends, the company moves to the first free plan offered. For each monthly allowance you can set an extra rate: with a rate, work carries on past the allowance and the extra is invoiced; without one, work stops at the allowance.
Tip: Switch on Razorpay first (Services → Payments, with the key ID, key secret and webhook secret), so companies can pay. In the Razorpay dashboard, add a webhook to https://api.<main site>/webhooks/razorpay (the api. address: the main address redirects, and Razorpay does not follow redirects) with the subscription and payment events.
Tip: Changing to another Razorpay account (for example from test keys to live keys) is fine: each plan is set up in the new account the first time a company chooses it.
In the guide: Platform console (Kombee staff) → Plans
Practice problems and moderation Ready
Open “Practice” to run the public practice area. The numbers at the top are practice’s own meter: code runs today against the daily ceiling, visitors today, and runs this month. Practice never uses a company’s allowance.
Full details
“Discussion waiting for a decision” lists new posts: click “Approve” to show one under its problem, or “Reject” (for full solutions, spam or anything rude).
Click “Add a problem”, or a problem’s title to edit it. For code, give each test the exact input and the exact output; tick “Example” for those to show (at least one), and keep at least one hidden. For multiple choice, put one answer per line and choose the right one; the write-up is shown as the explanation. New problems stay hidden until you click “Publish”; “Hide” takes one off the catalogue but keeps everyone’s progress.
In the guide: Platform console (Kombee staff) → Practice problems and moderation
Services and AI models Ready
Which outside services (email, video, AI) are used, and which AI models. “Stand-in (mock)” means nothing really leaves SetBenchmark: emails, for example, are kept for testing and never reach anyone.
Full details
The same works for every service: SMS and WhatsApp (MSG91 for SMS, Gupshup for WhatsApp, or both), payments (Razorpay), speech for voice interviews (Deepgram), video interviews (LiveKit Cloud) and the AI interviewer avatar (Tavus). Each asks for its own keys; SMS and WhatsApp also need a message template approved with the provider.
To send real email, open “Switch provider” under Email and choose Amazon SES, Resend, Mailjet or SMTP. Fill in its key and the From address (on a domain you have verified with that provider), then “Switch now”. It takes effect at once, without a restart. Keys go to the secret store and never show again.
AI interviewer avatar: choose Tavus, paste Kombee’s Tavus key and the Face ID of the face to show, and set the longest interview in minutes (45 when blank). Leave PAL ID blank and press “Check”: an echo PAL is made and its id shown; paste it into PAL ID and click “Save settings”, so the same one is reused. Kombee’s Tavus account pays per minute; each room ends by itself at the limit, and minutes show under each company’s usage as interview minutes. Keep the limit within the longest conversation your Tavus plan allows (Builder: 15 minutes). If an interview runs past it, the face goes away and the rest of the questions are read aloud.
To change a setting of the provider that is on (a new key, a PAL ID, a limit), open “Switch provider”: it is listed first, marked “(on now)”, with its settings filled in. Change what you need and click “Save settings”. A key left blank keeps the one saved.
“Check” under Email sends a test email to you, or to the address you type in the box. If it fails, the message says why (for example, the domain isn’t verified yet). Every switch and check is listed under “History”.
Tip: Emails not reaching candidates or staff? Check that Email isn’t on the stand-in, then use “Check”.
Tip: Amazon SES needs an access key on Google Cloud servers; leave the key blank on AWS servers, which use their own role.
Tip: No email domain of your own yet? Choose SMTP and use a Gmail account: Server smtp.gmail.com, Port 465, Username the Gmail address, Password a 16-letter app password (Google account → Security → 2-Step Verification on → App passwords), From address the same Gmail address. Gmail sends about 500 emails a day, so use it for demos and small volumes. Microsoft 365 (smtp.office365.com, port 587) and Zoho work the same way.
In the guide: Platform console (Kombee staff) → Services and AI models
Incidents Ready
Code the sandbox stopped, across every company.
In the guide: Platform console (Kombee staff) → Incidents
Support and status Ready
Answer companies’ questions, and post notices on the public status page, such as planned maintenance.
In the guide: Platform console (Kombee staff) → Support and status