# Exploring the npm registry API In this post, we will learn how to use the REST API provided by the [npm registry](https://www.npmjs.com/) to programmatically discover public Javascript packages and retrieve their metadata. ## What is npm? The term `npm` refers both to: - The `npm` CLI tool installed by default with [Node.js](https://nodejs.org/) on your machine - The [npm registry](https://www.npmjs.com/), an online service which collects more than 1.6M public Javascript packages For example, when running the `npm install react` command in your Javascript project, you are downloading the `react` package from the online npm registry. In this post, we are interested in the online service and its public API. ## What is the npm registry API? While many people regularly use [npm's website](https://www.npmjs.com/) to discover packages, only a few know that npm also provides a [public REST API](https://github.com/npm/registry/blob/master/docs/REGISTRY-API.md) accessible at [registry.npmjs.org](https://registry.npmjs.org/). This API provides methods to: - Get information about the registry itself - Get all available information about a specific package - Get information about a specific version of a package - Search packages by text - Count the number of downloads for packages We can call these methods by: - Sending HTTP requests to the correct endpoints - Using a [fully typed](https://www.jsdocs.io/package/query-registry) package I developed named [query-registry](https://www.npmjs.com/package/query-registry) ## Before we start You can follow along online on [this RunKit notebook](https://runkit.com/velut/60dde43a820278001aacda44) or on your machine by installing the `isomorphic-unfetch` and `query-registry` packages as follows: ```bash npm install isomorphic-unfetch query-registry ``` You can also use as references the [official API specification](https://github.com/npm/registry/blob/master/docs/REGISTRY-API.md) and the [documentation for query-registry](https://www.jsdocs.io/package/query-registry). Finally, you can explore the API and its responses in your browser by going to [registry.npmjs.org](https://registry.npmjs.org/). ## Example 1: Get information about the registry itself ### Endpoint If we want to know more about the underlying database used by the registry, we can send a `GET` request to the `/` endpoint, that is [`https://registry.npmjs.org/`](https://registry.npmjs.org/). ### With `fetch` ```javascript async function example1WithFetch() { const endpoint = "https://registry.npmjs.org/"; const res = await fetch(endpoint); const data = await res.json(); console.log(data); } ``` ### With `query-registry` ```javascript async function example1WithQueryRegistry() { const data = await queryRegistry.getRegistryMetadata(); console.log(data); } ``` ### Response We receive a response containing information about the registry's database, including its name and some interesting attributes, as shown below: ```json { "db_name":"registry", "engine":"couch_bt_engine", "doc_count":2226548, "doc_del_count":334, "update_seq":5769731, "purge_seq":0, "compact_running":false, "sizes":{ "active":57693928578, "external":132154863659, "file":58937123056 }, "disk_size":58937123056, "data_size":57693928578, "other":{ "data_size":132154863659 }, "instance_start_time":"1624686290809498", "disk_format_version":7, "committed_update_seq":5769731, "compacted_seq":5729968, "uuid":"964c127ddcbbd59982db296a0f9e8a56" } ``` ## Example 2: Get all available package metadata ### Endpoint If we want to get a *packument* (package document) containing all the information available on a package, we can send a `GET` request to the `/` endpoint, for example [https://registry.npmjs.org/react](https://registry.npmjs.org/react) or [https://registry.npmjs.org/@types/node](https://registry.npmjs.org/@types/node). ### With `fetch` ```javascript async function example2WithFetch(name) { const endpoint = `https://registry.npmjs.org/${name}`; const res = await fetch(endpoint); const data = await res.json(); console.log(data); } ``` ### With `query-registry` ```javascript async function example2WithQueryRegistry(name) { const data = await queryRegistry.getPackument({ name }); console.log(data); } ``` ### Response We receive a response containing all the data associated to a package, including its ID, name, description, author, license and manifests for each published version. ```json { "_id": "react", "_rev": "1684-29eba7dd741dee3c165b86b7e4f63461", "name": "react", "description": "React is a JavaScript library for building user interfaces.", "dist-tags": {…}, "versions": {…}, "maintainers": […], "time": {…}, "repository": {…}, "readme": "", "readmeFilename": "", "homepage": "https://reactjs.org/", "keywords": […], "bugs": {…}, "users": {…}, "license": "MIT" } ``` ## Example 3: Get information about a specific version of a package ### Endpoint If we want to get a *package manifest* containing information about a specific version of a package, for example `react@17.0.2` or `@types/node@15.14.0`, we can send a `GET` request to the `//` endpoint, for example [https://registry.npmjs.org/react/17.0.2](https://registry.npmjs.org/react/17.0.2) or [https://registry.npmjs.org/@types/node/15.14.0](https://registry.npmjs.org/@types/node/15.14.0). ### With `fetch` ```javascript async function example3WithFetch(name, version) { const endpoint = `https://registry.npmjs.org/${name}/${version}`; const res = await fetch(endpoint); const data = await res.json(); console.log(data); } ``` ### With `query-registry` ```javascript async function example3WithQueryRegistry(name, version) { const data = await queryRegistry.getPackageManifest({ name, version }); console.log(data); } ``` ### Response We receive a response containing data that describes a published version of a package. This data consists of the contents of `package.json` at publishing time plus some additional attributes added by the registry. ```json { "name": "react", "description": "React is a JavaScript library for building user interfaces.", "keywords": […], "version": "17.0.2", "homepage": "https://reactjs.org/", "bugs": {…}, "license": "MIT", "main": "index.js", "repository": {…}, "engines": {…}, "dependencies": {…}, "browserify": {…}, "_id": "react@17.0.2", "_nodeVersion": "15.11.0", "_npmVersion": "7.6.0", "dist": {…}, "_npmUser": {…}, "directories": {}, "maintainers": […], "_npmOperationalInternal": {…}, "_hasShrinkwrap": false, } ``` ## Example 4: Search packages by text ### Endpoint If we want to search packages by text, we can send a `GET` request to the `/-/v1/search?text=` endpoint, for example [https://registry.npmjs.org/-/v1/search?text=react](https://registry.npmjs.org/-/v1/search?text=react). We can also use special keyword parameters in our text query to improve our results. For example, to find packages that I published we can use the `author:velut` keyword parameter like this: [https://registry.npmjs.org/-/v1/search?text=author:velut](https://registry.npmjs.org/-/v1/search?text=author:velut). The [official API specification](https://github.com/npm/registry/blob/master/docs/REGISTRY-API.md#get-v1search) contains the full list of supported search criteria. ### With `fetch` ```javascript async function example4WithFetch(text) { const endpoint = `https://registry.npmjs.org/-/v1/search?text=${text}`; const res = await fetch(endpoint); const data = await res.json(); console.log(data); } ``` ### With `query-registry` ```javascript async function example4WithQueryRegistry(text) { const data = await queryRegistry.searchPackages({ query: { text } }); console.log(data); } ``` ### Response We receive a response containing a list of packages matching our query inside the `objects` attribute. Each package comes with a small number of important attributes, including `name` and `version`, plus some score values for the package itself and for its relevance to our query. ```json { "objects": [ { "package": { "name": "react", "scope": "unscoped", "version": "17.0.2", "description": "React is a JavaScript library for building user interfaces.", "keywords": ["react"], "date": "2021-03-22T21:56:19.536Z", "links": { "npm": "https://www.npmjs.com/package/react", "homepage": "https://reactjs.org/", "repository": "https://github.com/facebook/react", "bugs": "https://github.com/facebook/react/issues" }, "publisher": { "username": "…", "email": "…" }, "maintainers": [ { "username": "…", "email": "…" }, { "username": "…", "email": "…" } ] }, "score": { "final": 0.5866665170132767, "detail": { "quality": 0.5246016720020373, "popularity": 0.8931981392742823, "maintenance": 0.3333333333333333 } }, "searchScore": 100000.63 } ], "total": 164637, "time": "Fri Jul 02 2021 13:13:14 GMT+0000 (Coordinated Universal Time)" } ``` ## Example 5: Count downloads for a package ### Endpoint If we want to count the number of downloads for a package in a given time period, we can send a `GET` request to a slightly different API endpoint at `https://api.npmjs.org/downloads/point//`, for example [https://api.npmjs.org/downloads/point/last-week/react](https://api.npmjs.org/downloads/point/last-week/react). Supported time periods include `last-day`, `last-week`, `last-month` and `last-year`. The [download counts API](https://github.com/npm/registry/blob/master/docs/download-counts.md) also provides other methods to count downloads for packages and for the whole registry. ### With `fetch` ```javascript async function example5WithFetch(name, period) { const endpoint = `https://api.npmjs.org/downloads/point/${period}/${name}`; const res = await fetch(endpoint); const data = await res.json(); console.log(data); } ``` ### With `query-registry` ```javascript async function example5WithQueryRegistry(name, period) { const data = await queryRegistry.getPackageDownloads({ name, period }); console.log(data); } ``` ### Response We receive a simple response containing the package's name, its total number of downloads and the start and end dates for the selected time period. ```json { "downloads": 10889040, "start": "2021-06-25", "end": "2021-07-01", "package": "react" } ``` ## Bonus: Using a registry mirror ### Why use a mirror? Sometimes we may want to use a proxy or mirror of the npm registry instead of the original registry itself. For example, [Cloudflare](https://www.cloudflare.com/) provides a mirror at [https://registry.npmjs.cf](https://registry.npmjs.cf/) with [CORS](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS) enabled, allowing us to query the registry directly from the browser or client-side applications. For example, try pasting this snippet in your browser's console: ```javascript fetch("https://registry.npmjs.org/react").then(res => res.json()).then(console.log) ``` It should fail with a CORS error because it's using the original registry. Instead, the following snippet should work because it's using Cloudflare's registry mirror. ```javascript fetch("https://registry.npmjs.cf/react").then(res => res.json()).then(console.log) ``` ### Endpoint We can use the same endpoints available on [registry.npmjs.org](https://registry.npmjs.org/) provided that they are supported by the chosen mirror registry. ### With `fetch` ```javascript async function bonusWithFetch(name) { const endpoint = `https://registry.npmjs.cf/${name}`; const res = await fetch(endpoint); const data = await res.json(); console.log(data); } ``` ### With `query-registry` ```javascript async function bonusWithQueryRegistry(name, registry) { const data = await queryRegistry.getPackument({ name, registry }); console.log(data); } ``` ### Response The responses should be the same as the ones provided by the original npm registry, maybe slightly delayed due to the mirroring process. ## Conclusion In this post, we have learnt what is npm, how we can use its public API to discover and analyze public Javascript packages and how we can take advantage of API client wrappers such as [query-registry](https://www.npmjs.com/package/query-registry) and registry mirrors such as [https://registry.npmjs.cf](https://registry.npmjs.cf/) to improve our interactions with this API both in server-side and client-side Javascript applications. Be sure to leave a comment if you have any doubts or if you end up building something interesting using this lesser known but powerful API. If you liked this article and want to know when I post more, you can [follow me on Twitter](https://twitter.com/EdoardoScibona). ## Credits - Cover photo by Ricardo Gomez Angel on Unsplash