On this page
- Project-Code-Challenge-Specific Details
- Project Code Challenge Preview UI
- Editor UI
- Saving & Publishing
- Running Code
- Shortcut Key Commands:
- Run Fileset
- Challenge Config
- Description & Settings
- Run Configuration
- Project Files
- Reference Solution Files
- Web IDE Tools
- Run Output
- Web Preview
- Files Sent to Runner
- Initial Layout
- IDE Settings
- IDE Help
Project Code Challenges
Project code challenges allow you to deeply test a candidate's skill using complete, realistic code bases. Unlike our classic code challenges, these can only be created against a single language, but the advantage is your candidate is able to come up with more advanced solutions that show off their experience.
Project code challenges have many files per challenge, which you can view inside the challenge preview.
Solving Project Code Challenges
The workflow for solving a project code challenge is:
- The candidate reviews the instructions, and provided setup code.
- They edit their solution using the online web IDE, modifying editable files, or adding new ones as necessary.
- For front-end web challenges, candidates can use the WEB PREVIEW to see real-time rendering for their solution while editing.
- At any time, they can use the RUN TESTS button to validate their solution. They can edit the provided test cases if they want to perform additional testing.
Using the SUBMIT SOLUTION button, their code is submitted against the submission test cases. Unlike with classic code challenges, this result is never shown to the candidate.
- Even if they don't click SUBMIT SOLUTION, Qualified will still detect when they are done with the challenge and process their solution.
- While they can't review the submission results, they are allowed to go back and edit their solution and re-submit it as long as the assessment itself has not been submitted. This could be if, for example, they had gotten stuck but thought of a solution and want to complete the challenge.
Project Code Challenge Preview UI
You can open the solver for project challenges either by previewing an entire assessment, or using the PREVIEW button on a challenge.
This solver gives you the ability to test writing solution code, add or modify test cases for validating the solution, and preview the output. As it's a preview, there is also a button to preview the submission test cases, but these will never be shown to the candidate.
Within the preview window, nothing is saved, this is purely a way to quickly view how a challenge works.
Creating Project Code Challenges
You can create a challenge by using the CREATE A CHALLENGE menu in the Challenges sidebar, or you can use the EDIT or DUPLICATE buttons to modify an existing one.
The project editor is more advanced than the classic code editor, and works more like a traditional IDE. You can open and close various pages and files, and re-arrange the UI by dragging tabs around.
You can preview a challenge before saving by using the PREVIEW button, or pressing ⌘CTRL + P. This will let you see how the challenge works as a candidate, including previewing scoring results. Pressing ⌘CTRL + P again will return you to the editor.
Saving & Publishing
Once you are comfortable with the challenge you have entered, you can choose to SAVE & PUBLISH the challenge immediately. If you aren't ready to exit the editor, you can use the Save Changes option under the save button menu.
If you don't want to publish the challenge, you can keep it in the Draft state by using Save Draft under the save button menu.
Publishing the challenge will also validate the challenge. This means each language will be submitted and run against your reference solution. This helps to ensure the quality of the challenge, but also enables our system to gather important baseline information, such as the total number of tests and other basic information about the challenge.
The editor can run your code for previewing output against either the candidate tests (those shown to the candidate) or the submission tests (those used for scoring). Use the RUN CANDIDATE TESTS and RUN SUBMISSION TESTS buttons respectively.
Shortcut Key Commands:
- ⌘CTRL + ': Run the candidate tests.
- ⌘CTRL + Enter: Run the submission tests.
Normally, you'll want to run the tests against the Reference fileset. This will allow you to test that your tests are all working as expected. Occasionally, you'll want to ensure the setup files don't pass the tests. To do this, switch the Run Fileset mode to Project (Setup), which will not include any reference files in the solution.
Description & Settings
This page provides all the general information about the challenge, such as title, summary, difficulty, and estimated time.
If you've imported a library challenge, it will be linked to the library version, and other options won't be editable until you unlink it.
This page provides a side-by-side Markdown editor that allows you to provide detailed instructions to the developer.
Allows you to view the environment and adjust some properties of the runtime configuration for this challenge, such as enabled services or paths that are ignored during submission.
These are all the base files that make up a challenge. These are what are provided to the candidate so they can solve the challenge.
Reference Solution Files
Reference solution files allow the code runner to validate your tests, as well as learn about the expected output from the tests. Any files provided in here are treated as files provided by a candidate, overriding (when allowed) the project files above.
If you want to preview which files are sent to the runner at any time, open the Files Sent to Runner tab.
Web IDE Tools
Open to see the output from running your code. It will automatically be opened if you use the RUN buttons.
For challenges that target front-end web development, we provide a real-time, in-IDE preview panel. You and the candidate can use this to see how a solution renders, enabling visualization of HTML5 UIs such as those generated by React, Vue and Angular.
Please note that Web Previews require a somewhat specific project layout, so you may be restricted in how your challenge is designed. Using one of the Qualified templates is the best way to get started with your front-end challenge.
Web preview is sometimes confused with the test suite evaluated on Qualified's server upon submission. Web preview and the server are separate execution environments defined by their own
package.json files. This means an app appearing to function correctly in web preview may not necessarily pass the submitted test suite.
To help guide candidates and students, we recommend including a brief note explaining that the web preview is not used for testing purposes.
Files Sent to Runner
This tab shows a listing of which files that will be sent to the runner based on which mode in use (Run Fileset and Candidate vs Submission tests).
Use this tab to configure the default layout a candidate will see when opening a challenge for the first time. (Once they've started, we'll save their tab layout for them.)
You can drag-and-drop files & IDE tools between the sections to configure which ones are opened by default, as well as the initial order. The first item in each section will be selected by default.
Configure the IDE to your liking here.
Inline help for the IDE.