Posted inProgramming

Stop Copy-Pasting: Automate PyPI and npm Releases with GitHub Actions

Programming tutorial - IT technology blog
Programming tutorial - IT technology blog

The Maintenance Trap: Why Copy-Pasting Code Kills Productivity

Most developers eventually hit a wall where they grow tired of copying the same helper functions between projects. You might write a perfect utility for date formatting or a custom API wrapper. Two weeks later, you start a new project and find yourself digging through old folders to find that specific file. You copy it over, but then you find a bug. Now you have to fix it in both places. Repeat this across ten projects, and you are no longer a developer—you are a manual sync engine.

I recently worked with a team of 12 where every microservice had its own version of a ‘logger’ utility. When we needed to update the log format to satisfy a new security requirement, we had to open 15 different repositories. We spent 40 hours of repetitive manual work just to apply a 10-line code change. It was a massive waste of engineering talent.

Root Cause: The Friction of Package Distribution

Why do we keep copy-pasting instead of building a library? Usually, it is because the initial setup feels heavy. Many developers struggle with three specific hurdles:

  • Tooling Confusion: Deciding between setup.py, Poetry, or Hatch for Python, or managing complex exports in a JavaScript package.json.
  • Stability Risks: The fear that a small change might break downstream projects without a robust testing suite.
  • Manual Release Stress: The anxiety of accidentally publishing a broken version or leaking sensitive API keys during a local terminal upload.

Without a standardized workflow, the ‘cost’ of creating a library feels higher than the ‘cost’ of copy-pasting. However, the long-term technical debt is always more expensive.

Evaluating Your Sharing Options

Before jumping into automation, let’s look at how most teams try to solve the sharing problem:

  1. Git Submodules: You link one repo inside another. This sounds efficient but managing versions is often a nightmare. It frequently breaks CI/CD pipelines when permissions aren’t perfectly synced.
  2. Internal Shared Folders: This only works if everyone is on the same local network. It fails the moment you need to scale or share code with the broader community.
  3. Public Registries (npm and PyPI): This is the industry standard. It provides Semantic Versioning (SemVer), easy installation via pip or npm, and automated dependency management.

The real efficiency gain comes from automating the release. You should never have to run npm publish or twine upload from your personal laptop again.

Step 1: Structuring Your Project for Success

A library needs a specific layout to be recognized by package managers. Here is a clean structure for cross-platform projects.

For a Python Library (PyPI)

Modern Python development uses pyproject.toml. This file follows PEP 517 and PEP 518 standards, replacing the fragmented setup.py and requirements.txt approach.

my-python-lib/
├── src/
│   └── my_library/
│       ├── __init__.py
│       └── core.py
├── tests/
│   └── test_core.py
├── pyproject.toml
├── README.md
└── LICENSE

Your pyproject.toml defines your build system and metadata:

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "my-awesome-lib"
version = "0.1.0"
description = "A high-performance utility for data processing"
requires-python = ">=3.8"
authors = [{ name = "Your Name", email = "[email protected]" }]

For a JavaScript/TypeScript Library (npm)

For npm, the package.json is the core. I recommend using TypeScript for libraries to provide users with automatic type definitions.

my-js-lib/
├── src/
│   └── index.ts
├── dist/
├── tests/
│   └── index.test.ts
├── package.json
├── tsconfig.json
└── README.md

In package.json, specify the files array to keep your package slim. Only include the compiled dist folder, not your source code:

{
  "name": "@your-username/my-js-lib",
  "version": "1.0.0",
  "main": "./dist/index.js",
  "types": "./dist/index.d.ts",
  "files": ["dist"],
  "scripts": {
    "build": "tsc",
    "test": "vitest run"
  }
}

Step 2: Writing Reliable Tests

You cannot publish a library without tests. If your library breaks, you break every application that depends on it. I have used this approach in production environments where stability is non-negotiable. For Python, use pytest. For JavaScript, vitest is currently the fastest and most modern choice. Don’t chase 100% coverage. Instead, focus on testing the public API: if a user inputs X, do they always receive Y?

# Simple Python test example
def test_addition():
    from my_library.core import add
    assert add(2, 3) == 5

Step 3: Automating the Release with GitHub Actions

The automation kicks in whenever you create a new “Release” tag on GitHub. This ensures a clean build environment and removes the “works on my machine” excuse.

The PyPI Workflow (Using Trusted Publishers)

Create .github/workflows/pypi-publish.yml. PyPI now supports “Trusted Publishing” via OIDC. This means you don’t need to store a password or token in GitHub Secrets.

name: Publish to PyPI
on:
  release:
    types: [published]
jobs:
  deploy:
    runs-on: ubuntu-latest
    permissions:
      id-token: write
    steps:
      - uses: actions/checkout@v4
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'
      - name: Build and Publish
        run: |
          pip install build
          python -m build
      - uses: pypa/gh-action-pypi-publish@release/v1

The npm Workflow

Create .github/workflows/npm-publish.yml. You will need to add an NPM_TOKEN to your GitHub Repository Secrets for this to function.

name: Publish to npm
on:
  release:
    types: [published]
jobs:
  publish:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20.x'
          registry-url: 'https://registry.npmjs.org'
      - run: npm ci
      - run: npm run build
      - run: npm publish
        env:
          NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

Best Practices for Maintenance

Publishing the code is only half the battle. To make your library useful to others, keep these four principles in mind:

  • Semantic Versioning: Use MAJOR.MINOR.PATCH. If you change a function name that breaks existing code, increment the Major version.
  • The README is your UI: A library without a README effectively doesn’t exist. Include a 5-line code snippet that users can copy to see immediate results.
  • Choose a License: Use MIT or Apache 2.0 for maximum adoption. Many companies are legally prohibited from using code that lacks a clear LICENSE file.
  • Keep Dependencies Slim: Every library you add becomes a burden for your users. If you can write a 10-line function yourself, avoid adding a 200KB dependency.

Moving your code to a global registry changes your perspective on development. It forces you to design cleaner interfaces and write better documentation. Once your GitHub Actions pipeline is live, releasing a new version is as simple as clicking a button. This workflow eliminates the friction of sharing and lets you focus on building features instead of managing files.

Share:
Tags: