Developers Italia provides a catalog of Free and Open Source software aimed to Public Administrations.

This crawler retrieves the publiccode.yml files from the repositories of publishers found in the Developers Italia API.

The crawler can either run manually on the target machine or it can be deployed from a Docker container.

1. Rename config.toml.example to config.toml and set the variables

   NOTE: The application also supports environment variables in substitution to config.toml file. Remember: "environment variables get higher priority than the ones in configuration file"

2. Build the binary with go build

You can build the Docker image using

docker build .

or use the image published to DockerHub:

docker run -it italia/publiccode-crawler

Gets the list of publishers from https://api.developers.italia.it/v1/publishers and starts to crawl their repositories.

Gets the list of publishers in publishers*.yml and starts to crawl their repositories.

• crawler download-publishers downloads organizations and repositories from the onboarding portal repository and saves them to a publishers YAML file.

• developers-italia-api: the API used to store the results of the crawling
• publiccode-parser-go: the Go package for parsing publiccode.yml files

Developers Italia is a project by AgID and the Italian Digital Team, which developed the crawler and maintains this repository.