Creating a developer portal with no budget (part 1/3)

illustrations illustrations illustrations illustrations illustrations illustrations illustrations

Published on 20 January 2022 by Andrew Owen

post-thumb
#Hugo #Docker #Git #DevPortal

In this three part series, I’ll outline how to create a fully featured dev portal for your Swagger or OpenAPI 3.0 content without spending a dime. You can read part two here and part three here.

Part I: Backend

A dev portal is just a website that presents API (and other) docs to developers. It can be as simple as a static web page or as complex as you want to make it. In this series, we’ll make use of the following back-end technologies:

  • Static Site Generators
  • Source control
  • Containerization
  • Automation

In addition, you’ll inevitably spend a fair amount of time using a browser. For better or worse, Chrome is the new standard.

Static Site Generators

Hugo

Hugo is a type of web server known as a static site generator (SSG). It’s lightweight and fast and can serve dynamic and static content. You can test content locally before deploying it. For OpenAPI content, we’ll use the command line version of ReDoc to convert Swagger JSON files to a static HTML page that’s served by Hugo.

Install CLI tools (macOS)
  1. Homebrew is a package manager. From the Terminal, enter /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
  2. Git provides source control. From the Terminal, enter git and follow the instructions to install Xcode.
  3. Hugo is a static site generator. From the Terminal, enter brew install hugo.
  4. NPM is a package manager for JavaScript. From the Terminal, enter brew install nodejs. ReDoc renders an OpenAPI file into a static HTML page. From the Terminal, enter npm install -g redoc-cli@0.13.2. You can leave off the @<version_number>, but I’ve found npm doesn’t always get the latest version if you do. Ensure you’re using version 0.9.8 or later:redoc-cli --version\.
  5. (Optional) Swagger2PDF converts an OpenAPI JSON file into a static PDF document. From the Terminal, enter npm install swagger-spec-to-pdf.
Install CLI tools (Windows)
Requirements
Instructions
  1. Download and install Scoop.
  2. Git provides source control. From PowerShell, enter scoop install git.
  3. Hugo is a static site generator. From PowerShell, enter scoop install hugo.
  4. NPM is a package manager for JavaScript. From PowerShell, enter scoop install nodejs.
  5. ReDoc renders an OpenAPI file into a static HTML page. From PowerShell, enter npm install -g redoc-cli0.13.2. You can leave off the @<version number>, but I’ve found npm doesn’t always get the latest version if you do. Ensure you’re using version 0.9.8 or later:redoc-cli --version.
  6. (Optional) Swagger2PDF converts an OpenAPI JSON file into a static PDF document. From PowerShell, enter npm install swagger-spec-to-pdf.
  7. cURL is a command line data transfer tool. From PowerShell, enter scoop install curl.
Test a local copy of the dev portal

Navigate to the folder where you have cloned the Git repository.

  1. From the command line, enter hugo server.
  2. Copy the URL from the console output and paste it into the Chrome browser.
Add Google Analytics to a static HTML page

Paste the following text after the <head> tag:

    <!-- Global site tag (gtag.js) - Google Analytics --><script async src="https://www.googletagmanager.com/gtag/js? id=<yourID>"></script>

Source control

Git

If you’re generating comments from code, you’ll be working directly in the software repository. Typically, this is a Git repository. You should familiarize yourself with the processes for creating branches, creating pull requests, resolving conflicts and merging changes. There are a number of commercial hosting options including GitHub, GitLab and Bitbucket. Here we’ll look at Bitbucket.

Get the URL from Bitbucket
  1. In Chrome, navigate to the required repository.
  2. In the left icon menu, click Clone.
  3. Copy the URL.
Clone the repository and create a local branch
  1. Open PowerShell / Terminal and navigate to where your local repository folder. Example: ~/development/.
  2. Enter git clone and paste the path you copied from Chrome. This creates a local copy of the repository.
  3. In Visual Studio Code, open the repository’s folder.
  4. In the lower left corner of the window, click Branch.
  5. Enter a name in the box ( feature/apidocs-<api class name>) and select the branch to base it on. Example: development

You can now make your changes.

Publish changes
  1. In Visual Studio Code, in the left icon menu, click Source Control.
  2. Enter a short description of why you made the changes in the box.
  3. Click the Commit (tick) icon. If prompted to configure Visual Studio Code to automatically stage changes, you should do so.
  4. In the lower left corner of the window, click Publish Changes.
Create a pull request
  1. In Chrome, navigate to the Bitbucket dashboard.
  2. From the Repositories menu, select the repository you’re working with.
  3. From the left icon menu, click Create pull request.
  4. Select your local repository from the Source list.
  5. Confirm the Destination repository (typically, development) and click Continue.
  6. Enter a Description. This should contain any additional explanation for the change.
  7. Attach a copy of the ReDoc HTML file.
  8. Select Reviewers. Typically, this is pre-populated. You can start typing a name to find a user. Ensure the product owner is included.
  9. Click Create.

You’ll receive email notifications when the status of the pull request changes. Example: When the change is approved.

Resolve conflicts
  1. In your local branch: git pull origin master.
  2. Merge incoming code changes while retaining doc changes.
  3. git commit -am "<your commit message>"
  4. git push
Merge changes

In Chrome, navigate to the pull request and click Merge.

Delete branch

  1. After the change is successfully merged, from the left icon menu, click Branches.

  2. Locate the branch you were working in and from the Actions menu, select Delete

    branch.

  3. Click Delete to confirm your action.

Edit text in Bitbucket

While not directly related to APIs, you may on occasion have to modify text contained in a file in a Git repository stored in Bitbucket. You must have a Bitbucket account to edit the files.

  1. Navigate to the file in Bitbucket.
  2. Click Edit.
  3. Make your changes.
  4. Click Commit.
  5. (Optional) Enter a title in the Commit message box.
  6. Select the Create a pull request for this change check box.
  7. Click Commit.
  8. (Optional) Enter a Branch name. Example: feature/uitext.
  9. Click Create pull request.
  10. Enter a Description.
  11. Click Create.

When your pull request has been reviewed and there has been at least one successful build, providing that there are no merge conflicts, you can merge your change.

  1. Navigate to the pull requests.
  2. Locate your pull request and click its hyperlink.
  3. Click Merge.

If the merge was successful, you should now delete your branch.

  1. Navigate to the branches.
  2. Click your branch’s Actions button and select Delete branch, then click Delete.

Issue tracking

Track documentation tasks in Jira
  • Standalone documentation tasks should have the issue type Documentation.
  • Developer tasks that require documentation should have the label Documentation.
  • Documentation should be part of the definition of done.

Containerization

Containers are an operating system-level virtualization technology. Sun Microsystems’ Solaris Zones were an early implementation in 2004. But the technology was popularized by Docker containers, release in 2013, and Kubernetes container management, released the following year.

You don’t need containerization if you’re using a hosting solution such as Netlify. However, if you’re hosting your own site, then you should at least consider using containers.

Install Docker

If you deploy your dev portal in a Docker container, you can try it locally before you build your automation tool chain with Docker Desktop.

After installation, you can run Docker images locally. Example: docker run -d --name rabbitmq -p 15672:15672 -p 5672:5672 bitnami/rabbitmq:latest

Create a Dockerfile for Hugo

The easiest way to deploy the Hugo static site is in a Docker image, as defined by a Dockerfile:

  • FROM defines the base image (in the example Alpine, a lightweight Linux distribution).
  • COPY copies files and folders to the Docker image.
  • ARG specifies arguments for the Docker build command.
  • RUN executes commands.
  • EXPOSE informs a user about the ports used.
  • CMD specifies the component and its arguments to be used by the image.

Here’s an example:

    FROM alpine:3.8 as runner
    COPY . .
    ARG HUGO_VERSION=0.80.0
    RUN apk --no-cache add \ 
            curl \
            git \
        && curl -SL https://github.com/gohugoio/hugo/releases/download/v
    ${HUGO_VERSION}/hugo_${HUGO_VERSION}_Linux-64bit.tar.gz \
    		-o /tmp/hugo.tar.gz \
        && tar -xzf /tmp/hugo.tar.gz -C /tmp \
        && mv /tmp/hugo /usr/local/bin/ \
        && apk del curl \
        && rm -rf /tmp/*
    
    EXPOSE 80
    CMD hugo --renderToDisk=true --watch=true --port=80 --bind="0.0.0.0" --
    baseURL="${VIRTUAL_HOST}" server
Start Docker on login (macOS)

Some APIs may require middleware, for example a local RabbitMQ instance. It can be convenient to have Docker start the middleware when you log in. For example, on macOS:

  1. Open the Automator application.

  2. Select the Application type and click Choose.

  3. In the Actions menu, select Utilities > Run Shell Script.

  4. In the Run Shell Script section, select /bin/bash from the Shell menu.

  5. In the box, enter:

    cd /usr/local/bin
    while (! ./docker stats --no-stream ); do
    sleep 10
    done
    ./docker start rabbitmq

  6. From the File menu, click Save.

  7. Navigate to the Applications folder (or your user Applications folder).

  8. Enter StartRabbit in the Save As box and click Save.

  9. Open System Preferences and click Users & Groups.

  10. Select your user and click Login Items.

  11. Click Add (+).

  12. Navigate to the location where you saved the StartRabbit application and select it.

  13. Click Add.

  14. Close System Preferences.

The script will run the next time you log in. The path has to be changed because /usr/local/bin isn’t part of the path for startup scripts. The script waits for the Docker daemon to start before starting the RabbitMQ container. It does this by querying Docker until it gets a response. A spinning cog is displayed in the right menu while the script is running.

If you’re deploying more than one Docker instance, that’s the perfect time to start using Kubernetes.

Automation

You should automate your deployments so that when your code changes, your API docs are updated. When software is developed using continuous integration and deployment, it helps if the docs are deployed that way too. A popular open source solutions for this is Jenkins. But if you’re using GitHub, you can use GitHub Actions:

name: Build ReDoc HTML
on:
  push:
    paths:
    - 'static/apis/**.yaml'
  workflow_dispatch:
    jobs:
      redoc:
        runs-on: ubuntu-latest
        permissions:
          contents: write
        steps:
        - uses: actions/checkout@v2
        - name: Create local changes
          run: |
            npm i -g redoc-cli
            redoc-cli build static/apis/swagger.yaml -o static/apis/swagger.html            
        - name: Commit files
          run: |
            git config --local user.email "github-actions@users.noreply.github.com"
            git config --local user.name "github-actions"
            git add .
            git commit -a -m "Add changes"            
        - name: Push changes
          run: git push origin main