Installation¶
First you need to install pipenv, it will handle the virtual environment creation for the project in order to sandbox our Python environment, as well as manage the dependency installation, among other things.
Start all dependent services using docker-compose (this will start PostgreSQL, Opensearch 2, RabbitMQ and Redis):
$ docker-compose up -d
Note
Make sure you have enough virtual memory for Opensearch in Docker:
# Linux
$ sysctl -w vm.max_map_count=262144
# macOS
$ screen ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/tty
<enter>
linut00001:~# sysctl -w vm.max_map_count=262144
Next, bootstrap the instance (this will install all Python dependencies and build all static assets):
$ ./scripts/bootstrap
Next, create database tables, search indexes and message queues:
$ ./scripts/setup
Running¶
Start the webserver and the celery worker:
$ ./scripts/server
Start a Python shell:
$ ./scripts/console
Upgrading¶
In order to upgrade an existing instance simply run:
$ ./scripts/update
Testing¶
Run the test suite via the provided script:
$ ./run-tests.sh
By default, end-to-end tests are skipped. You can include the E2E tests like this:
$ env E2E=yes ./run-tests.sh
For more information about end-to-end testing see pytest-invenio
Documentation¶
You can build the documentation with:
$ pipenv run build_sphinx
Production environment¶
You can use simulate a full production environment using the
docker-compose.full.yml
. You can start it like this:
$ ./docker/build-images.sh
$ docker-compose -f docker-compose.full.yml up -d
$ ./docker/wait-for-services.sh --full
Remember to create database tables, search indexes and message queues if not already done:
$ docker-compose -f docker-compose.full.yml run --rm web-ui ./scripts/setup
In addition to the normal docker-compose.yml
, this one will start:
HAProxy (load balancer) – https://127.0.0.1 and http://127.0.0.1:8080
Nginx (web frontend)
UWSGI (application container)
Celery (background task worker)
Flower (Celery monitoring) – http://127.0.0.1:5555
Opensearch-dashboards (Opensearch inspection) – http://127.0.0.1:5601
RabbitMQ (message queue) – http://guest:guest@127.0.0.1:15672
If you update any of the static files, including any of the React code (or anything else touched by webpack), then you’ll need to make sure all the containers running off the app image get recreated in order to refresh the static_data volume:
$ export deps_ver=#latest version
$ docker-compose -f docker-compose.full.yml kill web-ui worker frontend
$ docker build --build-arg DEPENDENCIES_VERSION=$deps_ver . -t tdotdat
$ docker-compose -f docker-compose.full.yml up -d web-ui worker frontend
You’ll also need to remember to periodically prune dangling images.