The API documentation test tool makes it easy to validate that the Markdown-based API documentation matches a REST service implementation.
The toolset includes a command line and GUI application that can be used to perform for the following validations:
- Check for broken links in the documentation.
- Print resource and method definitions.
- Verify that the documentation is internally consistent:
- Check that defined resources and APIs that return these resources match.
- Check that example API responses are consistent with the resources they should return.
- Verify that a target REST service matches the API documentation:
- Check that requests and responses in the documentation match the service.
- Inject parameters into the API calls to the service.
- Publish documentation to an output folder.
To build the project, invoke either msbuild or xbuild depending on your
platform. This tool is compatible with Mono or .NET.
Note: After cloning and prior to the first build, make sure to run the two commands
git submodule init and git submodule update to fetch all the data from the markdowndeep submodule.
apidocs.exe [command] [options]
Available commands are:
print- Print files, resources, and methods discovered in the documentation.check-links- Verify that links in the documentation aren't broken.check-docs- Check for errors in the documentation's resources, requests, and response examples.check-service- Check for differences between the documentation and service responses to documented requests.publish- Publish the documentation into one of the supported output formats.
All commands have the following options available:
| Option | Description |
|---|---|
--path <path> |
Path to the root of the documentation set to scan. If missing, the current path is assumed. |
--short |
Print concise output to the console. |
--verbose |
Print verbose output to the console, including full HTTP requests/responses. |
--log <log_file> |
Log console output to a file. |
--parameters <params> |
A URL-encoded string containing key/value pairs. Allows additional parameters to be passed to the task. Currently used by the tagging feature to specify content to include. For more information see Markdown customizations. |
Print information about the source files, resources, methods, and requests that were parsed by the tool.
| Option | Description |
|---|---|
--files |
Output information about the files contained in the document set. |
--resources |
Output resource definitions read from the documentation. |
--methods |
Output method definitions read from the documentation. |
One of these three arguments is required to use the print command.
Check for broken links in the documentation.
No specific options are required. Using --verbose will include warnings about
links that were not verified.
Example: apidocs.exe check-links --path ~/github/api-docs --method search
The check-docs command ensures that the documentation is internally consistent.
It verifies that:
- The JSON examples are proper JSON
- The API methods that accept or return a specific resource type have valid request/response examples
- The metadata in the documentation is formatted properly
| Option | Description |
|---|---|
--method <method_name> |
Optional. Specify the name of a request method to evaluate. If missing, all methods are evaluated. |
Example: apidocs.exe check-docs --path ~/github/api-docs --method search
Check the documented requests and responses against an actual REST service. This option will load accounts from a configuration file (see below), environment variables or use the specified access-token and url parameters to determine how to make API calls with the target service.
Any scenario files that are contained within the documentation path will automatically be loaded and used by the check-service method.
| Option | Description |
|---|---|
--access-token "token" |
OAuth access token to use when calling the service. You may need to escape the token value by enclosing it in double quotes. |
--branch-name <branch> |
Optional. Specify the branch name that is the source of the documentation. Compares this name to the configuration file to see if the check-service command is allowed for this branch. |
--headers <headers> |
Optional. Enables adding additional headers to every API call made by check-service. The format should be a quoted string with a |
--ignore-scopes |
Optional. Disables using scopes to determine which methods are exercised on an account. |
--method <method_name> |
Optional. Check a single request/response method instead of everything in the documentation. |
--odata-metadata <value> |
Optional. Specify the value of the odata.metadata level that is provided in the Accept header. |
--pause |
Pause for a key press between API calls to the service to enable reading the responses. |
--scopes <scopes> |
Provide a command separated list of scopes which are provided by the command line accessToken. |
--url <url> |
Set the base URL for the service calls. |
Example:
apidocs check-service --method "search" --access-token "foo" --url https://example.org/v1.0
apidocs check-service --headers "If-Match: *|Application: apidocs-test-app" --odata-metadata "odata.metadata=none"
You can specify account information in a configuration file stored inside the
documentation set. Apidocs will look for any .json file that includes an
accounts property include an array of account objects. These accounts will
be used by the check-service command.
See more details about account configuration files..
Instead of using an access token on the command line or an account configuration file you can use the following environment variables to provide a refresh token and token service to generate access tokens. This enables the tool to be used in automation scripts and other scenarios where it may not be possible to provide an access-token in any other way.
| Variable name | Description |
|---|---|
| oauth-token-service | URL for the OAuth 2.0 token service to be used to retrieve an access token |
| oauth-client-id | Client ID that is passed to the token service |
| oauth-client-secret | Client Secret that is passed to the token service |
| oauth-redirect-uri | Redirect URI used to generate the refresh token |
| oauth-refresh-token | Refresh token that is used to generate an access token |
If these environment variables are set, it is not necessary to pass an access
token using the --access-token command line parameter. The tool will call the
token service to retrieve an access token when necessary.
The publish command uses the documentation to generate a new set of outputs.
See the documentation on publishing using APIDocs for more details.
See Markdown requirements for more details about requirements on the markdown source for documentation.
The tool also supports defining parameters for requests in a separate file. This information is loaded and used to make one or more requests to the service by substituting values for placeholders in the initial request.
For example, in a request for an item with a particular ID, you might write the request to look like this:
<!-- { "blockType": "request"; "name": "get-drive" } -->
GET /drives/{drive-id}
However, when the test tool makes the API call to the service, calling it verbatim would result in an error. Request parameters allow you to define one or more scenarios that are used to call the method.
A scenario can have one or more statically defined properties. It can also include an HTTP request and substitute one or more placeholder values with data from the response to that request.
The scenario file contains a single JSON array, with each member of the array conforming to this schema:
{
"name": "Copy test_copy_file to a new location",
"method": "copy-item",
"enabled": true,
"test-setup": [
{
"method": "upload-via-put",
"http-request": "PUT /drive/root:/test_copy_file.txt:/content\r\nContent-Type: application/octet-stream\r\n\r\nTest file that we will copy to another location",
"request-parameters":
{
"{path-to-file}": "/test_copy.file.txt",
"!body": "Test file that we will copy to another location",
"Content-Type:": "application/octet-stream"
},
"allowed-status-codes": [ 200 ],
"capture": {
"[source-file-id]": "$.id",
"[response-type]": "Content-Type:",
"[response-body]": "!body"
}
}
],
"request-parameters":
{
"{item-id}": "[source-file-id]"
},
"expectations":
{
"$.size": 123,
"content-type:" "application/json",
"$.id": ["12345", "67890"]
}
}| Property | Type | Description |
|---|---|---|
name |
string | The name of the scenario described. |
method |
string | The name of the method this scenario uses. Either defined in the documentation or a substitute name is auto-generated. |
enabled |
bool | Enable or disable the scenario. |
test-setup |
array | See below. |
request-parameters |
key-value pairs | Specify the key-value pairs for parameters for the request. The key is used as a placeholder name, and the value is subed into the placeholder. |
expectations |
key-value pairs | Specify the expected values in the final response. The keys are the same Capture Grammar defined for request-parameters. |
The test-setup property allows you to define an array of calls that are made before the actual test method is executed. This allows you to pull values from other requests and store them to be used in the test method call. This also allows you to chain together multiple calls from the documentation to enable testing complex scenarios, like fragment uploads.
Each object in the array of test-setup is a PlaceholderRequest instance.
| Property | Type | Description |
|---|---|---|
method |
string | The name of a method from the documentation that should be used as this test-setup call |
http-request |
string | Instead of specifying a method from the docs, you can input a raw HTTP request to be used. |
request-parameters |
key-value pairs | Specify the key-value pairs for parameters for the request. The key is used as a placeholder name, and the value is subed into the placeholder. |
allowed-status-codes |
array of int | Normally the request is considered failed of the response is anything other than 2xx. Use this to allow error codes and other responses to be considered valid. |
capture |
key-value pairs | Specify the key-value pairs of values that are read from this response and stored for another request under this scenario. Allows you to store values and use them in other requests under the same scenario. |
canned-request |
string | The name of the canned-request defined in the scenario file that is executed. This way common requests can be stored in one place instead of repeated throughtout the test files. |
Canned requests look just like a test setup method, but instead of being a scenario for a particular method are avaialble to be used from any scenario definition.
{
"canned-requests": [
{
"name": "create-photo-item",
"method": "upload-via-put",
"request-parameters": {
"{item-path}": "!random-filename-png",
"!body.base64": "iVBORw0KGgoAAAANSUhEUgAAAAUAAAAFCAYAAACNbyblAAAAHElEQVQI12P4//8/w38GIAXDIBKE0DHxgljNBAAO9TXL0Y4OHwAAAABJRU5ErkJggg=="
},
"capture": {
"[item-id]": "$.id"
}
}
]
}When specifying a placeholder name or value, the following syntax is used:
| Syntax | Example | Description |
|---|---|---|
| Curly Braces | {path-to-file} |
Find and update a value in the URL matching the full string. |
| Square Braces | [source-file-id] |
Look for a previous stored value that was output from a previous request within the same scenario. |
| JPath | $.id |
Replace a property value in the JSON body of the request. If the content-type of the request is not application/json an error will occur. |
| !body | !body |
Replace the content stream of the request with the provided value |
| !body.base64 | !body.base64 |
Replace the content stream of the request with a decoded byte stream of the base 64 value provided. |
| !url | !url |
Replace the URL for the request with the provided value. |
| Header: | Content-Type: |
Replace the value of a header with the specified value. Note the header name must end with a colon to be valid. |
| Random Filename | `!random-filename-png! | Returns a randomly generated value with a particular file extension that can be used to ensure tests don't interfere with each other. |
The key of anything in the capture node MUST be wrapped in square
brackets [foobar]. Otherwise the parameters will not be considered value.
The output-value grammar follows the same syntax as the placeholder grammar:
| Syntax | Example | Description |
|---|---|---|
| JPath | $.id | Read and store the value at the JPath |
| Header: | Content-Type: |
Read and store the value of the specified HTTP header |
| !body | !body | Read and store the complete body of the response |
The HTML-comment enclosed JSON object inside the documentation has the following properties defined:
{
"blockType": "unknown | resource | request | response | ignored | example | simulatedResponse",
"@odata.type": "resource name",
"optionalProperties": [ "prop1", "prop2" ],
"isCollection": false,
"collectionProperty": "value",
"isEmpty": false,
"truncated": true,
"name": "string name",
"expectError": false,
"nullableProperties": [ "prop3", "prop4" ]
}| Name | Value | Allowed blocks | Description |
|---|---|---|---|
| blockType | string | All | Describes the type of the json block proceeding the annotation. |
| @odata.type | string | All | Describes the name of the resource (either being defined, in the case of a resource block, or as the body type on a request/response block) |
| optionalProperties | array of strings | resource | An array of properties that are not required to be in the code block. |
| isCollection | boolean | response, example, simulatedResponse | Indicates that the block contains a collection of items that match the @odata.type schema. This is expected as an object with a single property that is an array of objects. |
| collectionProperty | string | response, example, simulatedResponse | Provides the name of the variable that contains the collection. Default value: value. |
| isEmpty | boolean | response, example, simulatedResponse | Indicates that the collection value is expected to be empty (or not). |
| truncated | boolean | response, example, simulatedResponse | Indicates that the block will not include all properties of the resource and that's not an error. Properties explicitly shown in the code block are always considered required when tested against the service. |
| name | string | request, example | Provides the name of the request method being defined. |
| expectError | boolean | response, example, simulatedResponse | Use this to indicate that instead of returning the normal response as defined, an error response will be returned. |
| nullableProperties | array of strings | response, example, simulatedResponse | Provide a list of properties that are allowed to have null values. By default, null values for a property will generate a warning. |
| Name | Description |
|---|---|
resource |
The json block describes a system resource (complex type) in the API. |
request |
The json block describes an HTTP request that can be made by clients. |
response |
The json block describes the HTTP response that is sent from the service. |
example |
An example of the JSON data that would be generated by the client or returned by the service, without being wrapped in an HTTP call. |
simulatedResponse |
Used for unit testing to simulate responses from the service. |
ignored |
No processing is done on the code block that follows. |
See OpenSourceNotes for more details about open source usage in markdown-scanner.
This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.