Home Web Front-end JS Tutorial Syncing an NPM Package Between Multiple Registries

Syncing an NPM Package Between Multiple Registries

Nov 25, 2024 am 07:53 AM

Syncing an NPM Package Between Multiple Registries

Photo by Nicolas Radzimski on Unsplash

The Problem

Our team relies on an internal GitLab NPM registry for managing packages. This works perfectly for internal distribution, but there are instances where a package needs to be shared with external collaborators. Syncing the package from the internal registry to an external one became a new challenge.

I set out to find a solution and, as usual, encountered a series of hurdles that turned this into another chapter in my developer struggles.

Attempt 1: Using HTTP Requests to Mimic NPM CLI

Initially, I thought I could handle this with raw HTTP requests, essentially mimicking what the NPM CLI does under the hood. The idea was simple: query the source registry for package metadata, fetch the tarball, and upload it to the target registry.

Why it failed:

• The process was too complex and fragile.
• It required manually constructing the entire package.json manifest for every version.
• Every small misstep led to broken packages.

Attempt 2: Leveraging npm-registry-sync

Next, I discovered npm-registry-sync, a library designed for syncing NPM registries. This tool almost solved the problem; it could monitor changes and replicate them across registries.

Why it didn’t work for me:

• It operates in “daemon mode,” polling for updates continuously.
• In a GitLab CI pipeline, I needed a one-off execution, controlled entirely by the pipeline — no background processes allowed.

The Winning Solution: The NPM CLI

Eventually, I realized I could stick to the tried-and-true NPM CLI. The steps were straightforward:

  1. Install the package locally from Registry A.
    1. Reconfigure NPM to point to Registry B.
    2. Publish the package to Registry B.

While this worked like a charm, there were a few extra steps needed to make it CI-friendly.


Configuring NPM for Multiple Registries

Managing registry configurations dynamically in a CI pipeline was a bit tricky.

Here’s how I solved it:

General Configuration

Using the npm CLI, you can set parameters for each registry:

npm config set "//my.awesome.registry.com:<parameter name>=<parameter value>"
Copy after login
Copy after login

Important Gotcha:
The URL in the config must exclude the protocol (https:).

Package-to-Registry Association

To associate a specific namespace or package with a registry:

npm config set "<your namespace>:registry" "<your registry url with https:>"
Copy after login
Copy after login

Handling Authentication in CI Pipelines

Some registries required a username/password combo, while others used tokens. Here’s what I learned:

Tokens

Tokens are straightforward, but ensure you strip the protocol when configuring the auth URL:

npm config set "//my.awesome.registry.com:<parameter name>=<parameter value>"
Copy after login
Copy after login

Basic Auth

Generating a basic auth hash (username:password) required attention to detail. In some distros, the base64 command has quirks that differ from others.

On macOS:

npm config set "<your namespace>:registry" "<your registry url with https:>"
Copy after login
Copy after login

Will give you (as expected):

npm config set "//my.registry.com:_authToken=<token>"
Copy after login

With docker linuxkit:

echo -n "<my username & password hash>" | base64
Copy after login

What is going on here? There is a line break!

On some distros, the wrap parameter has a default set to 76 chars for formatting private keys etc.

It works like this:

d2hhdCBhcmUgeW91IGRvaW5nIGhlcmU/IGdvdCB5YSEgc29tZSBtb3JlIHRleHQgdG8gbWFrZSB0aGlzIHJlYWxseSByZWFsbHkgbG9uZw==
Copy after login

The echo -n is very important. If omitted, echo will add a linebreak to the end of the string and this will manipulate your hash.

The finished script

Putting it all together, the script could look like this:

d2hhdCBhcmUgeW91IGRvaW5nIGhlcmU/IGdvdCB5YSEgc29tZSBtb3JlIHRleHQgdG8gbWFrZSB0
aGlzIHJlYWxseSByZWFsbHkgbG9uZw==
Copy after login

So you could use it like this:

echo -n "<my username & password hash>" | base64 --wrap 0
Copy after login

Key Lessons Learned

  1. Stick to Simple Tools:
    The npm CLI might not be fancy for this task, but it’s reliable and gets the job done.

  2. Mind the Details:
    Configuring authentication, especially with base64, can have subtle platform-specific quirks.

  3. Keep It CI-Friendly:
    Avoid solutions like daemons or background tasks when working in CI/CD pipelines. Keep the process under pipeline control.

Syncing npm packages between registries was a frustrating but rewarding learning experience. If you’re facing a similar challenge, I hope these lessons help you navigate the struggle with a bit more ease!

The above is the detailed content of Syncing an NPM Package Between Multiple Registries. For more information, please follow other related articles on the PHP Chinese website!

Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

Video Face Swap

Video Face Swap

Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Tools

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

Hot Topics

Java Tutorial
1664
14
PHP Tutorial
1267
29
C# Tutorial
1239
24
Demystifying JavaScript: What It Does and Why It Matters Demystifying JavaScript: What It Does and Why It Matters Apr 09, 2025 am 12:07 AM

JavaScript is the cornerstone of modern web development, and its main functions include event-driven programming, dynamic content generation and asynchronous programming. 1) Event-driven programming allows web pages to change dynamically according to user operations. 2) Dynamic content generation allows page content to be adjusted according to conditions. 3) Asynchronous programming ensures that the user interface is not blocked. JavaScript is widely used in web interaction, single-page application and server-side development, greatly improving the flexibility of user experience and cross-platform development.

The Evolution of JavaScript: Current Trends and Future Prospects The Evolution of JavaScript: Current Trends and Future Prospects Apr 10, 2025 am 09:33 AM

The latest trends in JavaScript include the rise of TypeScript, the popularity of modern frameworks and libraries, and the application of WebAssembly. Future prospects cover more powerful type systems, the development of server-side JavaScript, the expansion of artificial intelligence and machine learning, and the potential of IoT and edge computing.

JavaScript Engines: Comparing Implementations JavaScript Engines: Comparing Implementations Apr 13, 2025 am 12:05 AM

Different JavaScript engines have different effects when parsing and executing JavaScript code, because the implementation principles and optimization strategies of each engine differ. 1. Lexical analysis: convert source code into lexical unit. 2. Grammar analysis: Generate an abstract syntax tree. 3. Optimization and compilation: Generate machine code through the JIT compiler. 4. Execute: Run the machine code. V8 engine optimizes through instant compilation and hidden class, SpiderMonkey uses a type inference system, resulting in different performance performance on the same code.

Python vs. JavaScript: The Learning Curve and Ease of Use Python vs. JavaScript: The Learning Curve and Ease of Use Apr 16, 2025 am 12:12 AM

Python is more suitable for beginners, with a smooth learning curve and concise syntax; JavaScript is suitable for front-end development, with a steep learning curve and flexible syntax. 1. Python syntax is intuitive and suitable for data science and back-end development. 2. JavaScript is flexible and widely used in front-end and server-side programming.

JavaScript: Exploring the Versatility of a Web Language JavaScript: Exploring the Versatility of a Web Language Apr 11, 2025 am 12:01 AM

JavaScript is the core language of modern web development and is widely used for its diversity and flexibility. 1) Front-end development: build dynamic web pages and single-page applications through DOM operations and modern frameworks (such as React, Vue.js, Angular). 2) Server-side development: Node.js uses a non-blocking I/O model to handle high concurrency and real-time applications. 3) Mobile and desktop application development: cross-platform development is realized through ReactNative and Electron to improve development efficiency.

How to Build a Multi-Tenant SaaS Application with Next.js (Frontend Integration) How to Build a Multi-Tenant SaaS Application with Next.js (Frontend Integration) Apr 11, 2025 am 08:22 AM

This article demonstrates frontend integration with a backend secured by Permit, building a functional EdTech SaaS application using Next.js. The frontend fetches user permissions to control UI visibility and ensures API requests adhere to role-base

Building a Multi-Tenant SaaS Application with Next.js (Backend Integration) Building a Multi-Tenant SaaS Application with Next.js (Backend Integration) Apr 11, 2025 am 08:23 AM

I built a functional multi-tenant SaaS application (an EdTech app) with your everyday tech tool and you can do the same. First, what’s a multi-tenant SaaS application? Multi-tenant SaaS applications let you serve multiple customers from a sing

From C/C   to JavaScript: How It All Works From C/C to JavaScript: How It All Works Apr 14, 2025 am 12:05 AM

The shift from C/C to JavaScript requires adapting to dynamic typing, garbage collection and asynchronous programming. 1) C/C is a statically typed language that requires manual memory management, while JavaScript is dynamically typed and garbage collection is automatically processed. 2) C/C needs to be compiled into machine code, while JavaScript is an interpreted language. 3) JavaScript introduces concepts such as closures, prototype chains and Promise, which enhances flexibility and asynchronous programming capabilities.

See all articles