Introduction
Apigee is an API management platform that offers an integrated Developer Portal for documenting exposed endpoints in the API proxies. This enables developers to interact with an API easily and familiarize themselves without too much complexity.
All APIs are subject to change, and therefore, the Apigee Developer Portal needs to change to reflect these updates. However, manually updating the API specification becomes a complex task quickly.
This article explains how to automate the process of updating APIs in Apigee.
Prerequisites
- Apigee Developer Portal
Automated Updating of Apigee Developer Portal APIs
Three things are necessary to make an API spec appear in the Apigee Developer Portal:
- The Open API specification, referred to as the spec file for short. In our case, we are using the classic Pet Store.
- An API Product – this is a collection of APIs. Since the Apigee Developer Portal works with them, we will create one to display our APIs. For demonstration purposes, our API product will only have one API.
- An existing Developer Portal to publish or update the API Product and specification.
The sections below explain how to automate the process of updating APIs on Apigee.
Note: The scripts are available in phoenixNAP’s GitHub repo. However, we suggest using the image hosted on GitHub Packages, which contains all the scripts and setup needed to get you up and running.
Step 1: Uploading an API Specification
Currently, Apigee does not provide any means of uploading an API specification through the Admin API. Use the Apigee UI to add or update API specifications.
1. Select the Import File option to import the Pet Store specification. Have Chrome DevTools (or equivalent) open while doing so.
2. You should see the API specification (in this example, ‘petshop’) listed, as in the image below.
3. The API specification was successfully uploaded. Let’s see what happened in the background by referring to the Network tab in Chrome DevTools.
At most, three API calls are necessary to add an API spec:
POSTPUTGET
POST, PUT, and GET API Calls
Below is the first API call:
POST https://apigee.com/dapi/api/organizations/lukeb-eval/specs/doc{
"folder": "173137",
"kind": "Doc",
"name": "petstore"
}
folder: The ID of your organization’s spec folder. This will always be the same since each organization has one spec folder and it never changes.kind: In this case it will always beDocsince we’re uploading a document.name: The name of the API spec. Although it is not enforced, use a unique name, otherwise it can get confusing when viewing the API specs.
The outcome of this call is the creation of an empty API spec with the specified name. The expected response from Apigee is as follows:
{
"id": "299536",
"kind": "Doc",
"name": "petstore",
"created": "2020-06-05T13:24:07.977Z",
"creator": "/orgs/lukeb-eval",
"modified": "2020-06-05T13:24:07.977Z",
"permissions": null,
"self": "/organizations/lukeb-eval/specs/doc/299536",
"content": "/organizations/lukeb-eval/specs/doc/299536/content",
"contents": null,
"folder": "/organizations/lukeb-eval/specs/folder/173137",
"folderId": "173137",
"body": null,
"trashed": false
}
Make note of the id. You will use it in the following call to tell Apigee in what file we need to put our content in:
PUT https://apigee.com/dapi/api/organizations/lukeb-eval/specs/doc/299536/contentThe request body for this call is the whole API spec YAML. As a response, Apigee will reply with a 200 OK, signaling that the API spec was successfully uploaded.
The last call is:
GET https://apigee.com/dapi/api/organizations/lukeb-eval/specs/folder/homeIt produces the following response:
{
"id": "173137",
"kind": "Folder",
"name": "/orgs/lukeb-eval root",
"created": "2019-06-06T07:39:58.805Z",
"creator": "/orgs/lukeb-eval",
"modified": "2019-06-06T07:39:58.805Z",
"permissions": null,
"self": "/organizations/lukeb-eval/specs/folder/173137",
"content": null,
"contents": [{
"id": "299536",
"kind": "Doc",
"name": "petstore",
"created": "2020-06-05T13:24:07.977Z",
"creator": "/orgs/lukeb-eval",
"modified": "2020-06-05T13:24:08.740Z",
"permissions": null,
"self": "/organizations/lukeb-eval/specs/doc/299536",
"content": "/organizations/lukeb-eval/specs/doc/299536/content",
"contents": null,
"folder": "/organizations/lukeb-eval/specs/folder/173137",
"folderId": "173137",
"body": null,
"trashed": false
}],
"folder": null,
"folderId": null,
"body": null,
"trashed": false
}
At this point, we know that the spec files are available in our organization. This is the last call executed because Apigee needs to update the UI to show the latest specs after adding a new one.
This call is also executed if we just open the Specs tab in Apigee.
Having gone through the HTTP requests, we are now able to automate the creation or updating of a spec:
GET https://apigee.com/dapi/api/organizations/lukeb-eval/specs/folder/home– This provides us with thefolderId. We need it for the subsequent call. It also indicates whether the spec needs to be created or updated, depending on if it already exists.POST https://apigee.com/dapi/api/organizations/lukeb-eval/specs/doc– If the specification does not exist, we execute this call to create an empty file.PUT https://apigee.com/dapi/api/organizations/lukeb-eval/specs/doc/299536/content– Fill in the spec with the latest information. Depending on whether it was a create or an update request, theidof the spec can be retrieved from steps 1 or 2.
These steps are reflected in our automation script upload_spec.py. This script requires the following arguments to run successfully:
name– The name of the spec that will be uploaded to Apigee. If this is not unique, the spec with the same name in Apigee will be updated.file– The path of the spec file on your machine.org– The organization name in Apigee. https://apigee.com/organizations/johnd-eval/proxies has organization name johnd-eval.username– The username of the Apigee user the script uses to fetch an access token. This access token is used for all REST calls executed by the script. The access_token expires every 12 hours. Usually a dedicated automation user is used here.password– The password for the above-mentioned username.
To create or update the spec for the example discussed above, use:
upload_spec.py --name petstore --file petstore.yaml --org lukeb-eval --username $APIGEE_USER --password $APIGEE_PASSWORDA typical successful outcome, taken from our automation pipeline, would be as follows:
Step 2: Uploading an API Product
In our automation effort, we created upload_product.py, which leverages the Apigee Management API to create or update an API Product.
The following arguments are required for the script to run successfully:
file– A JSON file representing the API Product to be created. An example of how to create it can be found in the Create API Product or Update API Product section of the Apigee Management API documentation. Name is always mandatory in the JSON file since it is used to check if the API Product exists or not. If the API product exists, it is updated with the new JSON file.org– The organization name in Apigee.username– The username of the Apigee user the script uses to fetch an access token.password– The password for the above-mentionerd username.
It can be executed as follows:
upload_api_product.py --file product_settings.json --org lukeb-eval --username $APIGEE_USER --password $APIGEE_PASSWORDA typical successful outcome, taken from our automation pipeline, would be as follows:
Step 3: Uploading an API Spec to the Developer Portal
Updating an API spec does not mean that the Apigee Developer Portal will automatically show the most recent one, and Apigee does not offer this through their management API.
This can be done manually through the UI, so we need to investigate what API calls are used to create or update API documentation on the Developer Portal.
1. In the APIs section of your selected portal in Apigee, click the +API button:
2. Select the API Product and the API spec. Finally, add a name and description for the API, and click Finish.
3. Pressing the Finish button prompts the following HTTP call:
POST https://apigee.com/portals/api/sites/lukeb-eval-portal/apidocs{
"title": "Pet Store",
"description": "",
"edgeAPIProductName": "Pet Store",
"visibility": true,
"anonAllowed": true,
"requireCallbackUrl": false,
"specId": "petstore",
"specContent": "299536"
}4. To display the API spec in the portal, you need the following parameters:
edgeAPIProductName– The name of the previously created API product.specId– The name of the previously created spec.specContent– The file ID of the previously created spec.
Confusingly, the specId is actually the spec’s name and its uniqueness is not enforced by Apigee. In our automation script, we are assuming the spec is uniquely named, and this will help retrieve the specContent through the specId further on.
Let us consider what the Apigee UI does when a spec changes. If the Pet Store spec is updated, the following will show up on the APIs screen in the Developer Portal:
By clicking the Manage Spec Snapshot yellow image on the right, we get the following screen:
Click Update snapshot. That updates the Developer Portal with the latest spec.
Automating API Requests
To automate the process, we need three API requests:
GET https://apigee.com/portals/api/sites/lukeb-eval-portal/apidocsThe first, produces the following response:
{
"status": "success",
"message": "all visible apidocs returned",
"data": [{
"id": 57782,
"siteId": "lukeb-eval-portal",
"title": "Pet Store",
"description": "Pet Store Description",
"visibility": true,
"enrollment": null,
"apiId": "pet-store",
"edgeAPIProductName": "Pet Store",
"specId": "petstore",
"specContent": "299536",
"specTitle": null,
"snapshotExists": true,
"snapshotModified": 1591371293000,
"modified": 1591371290000,
"anonAllowed": true,
"imageUrl": null,
"snapshotState": "OK_DOCSTORE",
"requireCallbackUrl": false,
"categoryIds": [],
"productExists": true,
"specModified": null,
"snapshotOutdated": false,
"snapshotSourceMissing": false
}],
"request_id": "1309320113",
"error_code": null
}From an automation perspective, this request provides you with the Developer Portal’s ID and lists all the specs on the portal.
After getting the portal ID, we need the following API call:
PUT https://apigee.com/portals/api/sites/lukeb-eval-portal/apidocs/57782{
"specId": "petstore",
"imageUrl": null,
"visibility": true,
"anonAllowed": true,
"description": "Pet Store Description"
}The above-mentioned call is used to confirm the basic setup of the API on the portal. The ID 57782 at the end of the request URL is the ID of the Developer Portal.
Upon completion, another request is executed:
PUT https://apigee.com/portals/api/sites/lukeb-eval-portal/apidocs/57782/snapshotThis request has no body, and it instructs the Developer Portal to display the latest spec.
With all this information, we are now able to automate the portal spec deployment as follows:
1. Use GET https://apigee.com/dapi/api/organizations/lukeb-eval/specs/folder/home to make sure the spec we want to show exists.
2. Include GET https://apigee.com/portals/api/sites/lukeb-eval-portal/apidocs to get the portal ID and check if we need to create or update the spec in the portal.
- If you need to create:
POST https://apigee.com/portals/api/sites/lukeb-eval-portal/apidocs
- If you need to update:
PUT https://apigee.com/portals/api/sites/lukeb-eval-portal/apidocs/57782
PUT https://apigee.com/portals/api/sites/lukeb-eval-portal/apidocs/57782/snapshot
To this effect, the upload_portal_documentation.py automation script was created. This needs the following parameters:
file– JSON file representing the API Portal Documentation to be created. An example of the JSON is shown previously in this post. SinceorgnameandspecIdare provided as parameters, they should not be part of the JSON file, otherwise they will be overwritten.specContentis retrieved from thespecIdprovided, since we are working on the premise that a spec name will always be unique.portal– Full name of the portal where we want to create the documentation. e.g. If a portal is accessed through https://johnd-eval-test.apigee.io, the full name of the portal is johnd-eval-test.org– Same as in upload_spec.py.username– Same as in upload_spec.py.password– Same as in upload_spec.py.
It can be executed as follows:
upload_portal_documentation.py --file portal_documentation_setup.json --org lukeb-eval --spec_name petstore --portal portal --username $APIGEE_USER --password $APIGEE_PASSWORDA typical successful outcome, taken from our automation pipeline, would be as follows:
Step 4: CI/CD Pipeline Integration
We have these scripts in a Docker image so it is very easy to import and execute them in your CI/CD pipeline.
Let’s take GitLab for example:
apigee-spec-deploy:
image: ghcr.io/neveropen/apigee-automation:v1.0.0
stage: spec-deploy
variables:
SPEC_NAME: petstore
SPEC_PATH: petstore.yaml
APIGEE_PRODUCT: product.json
APIGEE_PORTAL_DOC: portal_documentation.json
script:
- /automation/upload_spec.py --name $SPEC_NAME --file $SPEC_PATH --org $ORG_NAME --username $APIGEE_USER --password $APIGEE_PASSWORD
- /automation/upload_api_product.py --file $APIGEE_PRODUCT --org $ORG_NAME --username $APIGEE_USER --password $APIGEE_PASSWORD
- /automation/upload_portal_documentation.py --file $APIGEE_PORTAL_DOC --org $ORG_NAME --spec_name $SPEC_NAME --portal $APIGEE_PORTAL --username $APIGEE_USER --password $APIGEE_PASSWORDWe have created a pipeline job called apigee-spec-deploy, which pulls the image apigee-automation from GitHub Packages, and executes the three python scripts we discussed here with the necessary parameters.
The GitLab job will stop executing if any script fails. If that occurs, a detailed error is provided in the output. This guarantees that whenever a script is executed, all the information it needs from the previous scripts will be available.
Conclusion
In this article, you have learned how to keep your Apigee developer portal APIs up to date by creating scripts that leverage calls done either in the Apigee UI or available in the Apigee Management API.
These scripts can easily be integrated into a CI/CD pipeline by using the Docker image provided. With this, your Apigee developer portal will always have the latest specs, without having to do it manually each time they change.
If you encounter any issues while implementing this, feel free to reach out to us on GitHub.
