The NWM API

Photo Credits: Unsplash

Introduction

From the very first time I used an article on this site to document one of my scripts, I've been thinking: wouldn't it be nice if I could wget or curl a script - and just the script, no explanations or html tags - from one of my articles?

Well, now I can. (And so can you, if you'd like.)

'Extract' API Usage

The API's root path is at https://nicholas-morris.com/api/extract or https://niwamo.com/api/extract (yes, I point two domains at this site). With just the base path, though, you'll receive a 404 - the API needs to be directed to specific content.

To form a proper request, check the URL path to the article you'd like to retrieve content from. To use this article as an example, check your current URI. (The path should be "/articles/nwm-api.") To retrieve this article's content via the API, simply append that to the base path.

Resource Path Example

Example

PS: nwm-example> Invoke-WebRequest -Uri "https://nicholas-morris.com/api/extract/articles/nwm-api"


StatusCode        : 200
StatusDescription : OK
Content           : {"data":[{"type":"thematicBreak","position":{"start":{"line":1,"column
                    ":1,"offset":0},"end":{"line":1,"column":4,"offset":3}}},{"type":"head
                    ing","depth":2,"children":[{"type":"text","value":"title: 'T...
RawContent        : HTTP/1.1 200 OK
                    content-security-policy: default-src 'self';  script-src 'self'
                    'unsafe-eval' 'unsafe-inline';  style-src 'self' 'unsafe-inline';
                    img-src * blob: data:;  media-src 'none';  connect-s...
Forms             : {}
Headers           : {[content-security-policy, default-src 'self';  script-src 'self'
                    'unsafe-eval' 'unsafe-inline';  style-src 'self' 'unsafe-inline';
                    img-src * blob: data:;  media-src 'none';  connect-src *;  font-src
                    'self';  frame-src], [referrer-policy,
                    strict-origin-when-cross-origin], [x-frame-options, DENY],
                    [x-content-type-options, nosniff]...}
Images            : {}
InputFields       : {}
Links             : {}
ParsedHtml        : mshtml.HTMLDocumentClass
RawContentLength  : 7494

Resource Path Example - Raw Content

Using the example above, you'll receive a JSON-formatted response with the requested content set as the value of a top-level element called "data." To retrieve raw content, simply add /raw to the URI.

Remember, though - if the data you've requested is a JSON array, you'll still get JSON. See the next section for more details on the return value.

Example

PS: nwm-example> Invoke-WebRequest -Uri "https://nicholas-morris.com/api/extract/articles/nwm-api/raw"


StatusCode        : 200
StatusDescription : OK
Content           : [{"type":"thematicBreak","position":{"start":{"line":1,"column":1,"off
                    set":0},"end":{"line":1,"column":4,"offset":3}}},{"type":"heading","de
                    pth":2,"children":[{"type":"text","value":"title: 'The NWM A...
RawContent        : HTTP/1.1 200 OK
                    content-security-policy: default-src 'self';  script-src 'self'
                    'unsafe-eval' 'unsafe-inline';  style-src 'self' 'unsafe-inline';
                    img-src * blob: data:;  media-src 'none';  connect-s...
Forms             : {}
Headers           : {[content-security-policy, default-src 'self';  script-src 'self'
                    'unsafe-eval' 'unsafe-inline';  style-src 'self' 'unsafe-inline';
                    img-src * blob: data:;  media-src 'none';  connect-src *;  font-src
                    'self';  frame-src], [referrer-policy,
                    strict-origin-when-cross-origin], [x-frame-options, DENY],
                    [x-content-type-options, nosniff]...}
Images            : {}
InputFields       : {}
Links             : {}
ParsedHtml        : mshtml.HTMLDocumentClass
RawContentLength  : 7494

Considerations for Raw Content on Windows

If you do use parameters to retrieve a single element, it will be returned as a raw string. This poses no issues with curl on Linux, but PowerShell's IWR will not like this - instead of showing you a string on the response's 'content' property, it will show you an array of the ordinal numbers of each character. (Annoying, right?) You can convert this back to text with the following:

[Text.Encoding]::Ascii.GetString((Invoke-WebRequest -Uri "https://nicholas-morris.com/api/extract/articles/nwm-api/raw").content)

(This is unnecessary if not filtering down to a single raw string.)

Return Value

If given no parameters, the API replies with a JSON array containing the full content of the requested article - including properties (e.g. "type"="heading" and "depth"=3). This array is the AST output of remark-parse. (I'll write a future article - or series of articles - about the site in the future, but briefly: each article is written in mdx and converted to web content with unified/remark/rehype.)

You could work with the AST output locally. On Windows I'd recommend something like:

Invoke-WebRequest -Uri "https://nicholas-morris.com/api/extract/articles/nwm-api" | ConvertFrom-Json

This will convert the response to a PowerShell object. On Linux, I'd recommend jq:

curl "https://nicholas-morris.com/api/extract/articles/nwm-api" | jq

However, this would require a decent amount of work: examining the data structure to figure out precisely what you'd like to extract. So, I've built in some optional parameters.

Optional Parameters


Type	Filter the response array to elements of the requested type, e.g. "code" or "paragraph"
Key	Retrieve only the specified value from each top-level child object (you will typically want "value" or "children")
Index	Return only the nth child element

Examples

Retrieve the Third Code Block from this Page

PowerShell

# Request
(Invoke-WebRequest -Uri "https://nicholas-morris.com/api/extract/articles/nwm-api?type=code&key=value&index=3").content

# Response
{"data":"[Text.Encoding]::Ascii.GetString((Invoke-WebRequest -Uri \"https://nicholas-morris.com/api/extract/articles/nwm-api/raw\").content)"}

Bash

# Request
curl "https://nicholas-morris.com/api/extract/articles/nwm-api?type=code&key=value&index=3"

# Response
{"data":"[Text.Encoding]::Ascii.GetString((Invoke-WebRequest -Uri \"https://nicholas-morris.com/api/extract/articles/nwm-api/raw\").content)"}

Retrieve the Third Code Block - As Raw Text