Skip to content

Commit

Permalink
Tidy up for Docker
Browse files Browse the repository at this point in the history
  • Loading branch information
JackGilmore authored Jun 9, 2023

Verified

This commit was signed with the committer’s verified signature.
bhaoo Bhao
1 parent c33bc5a commit bbcc4c8
Showing 1 changed file with 2 additions and 49 deletions.
51 changes: 2 additions & 49 deletions .github/workflows/pipeline.yml
Original file line number Diff line number Diff line change
@@ -38,67 +38,20 @@ jobs:
repository: 'OpenDataScotland/jkan'
path: 'jkan'
token: ${{ secrets.REPO_COMMIT_TOKEN }}
- name: Check files
run: ls -l -a

# Set up programming toolsets
# - name: Set up Python 3.9
# uses: actions/setup-python@v3
# with:
# python-version: '3.9'
# - name: Set up Python toolsets
# run: sudo apt-get install python3.9-dev pkg-config libcairo2-dev

# - name: Install dependencies for python scripts
# run: python -m pip install pandas markdown pyyaml beautifulsoup4 datefinder requests loguru

# Set up log file
# - name: Set up log file
# run: |
# touch log.json
# touch log.md
# echo '# pipeline error log' >> log.md
# echo '' >> log.md
# echo '## Unaccessible Webpages' >> log.md
# echo '' >> log.md
# echo '|URL | Error Code | Error Reason|' >> log.md
# echo '|--- | --- | ---|' >> log.md
# working-directory: ${{env.odbodsPath}}

# - name: Check files
# run: ls -l -a

# # Run main.py
# - name: Run Script main.py
# run: python main.py
# working-directory: ${{env.odbodsPath}}

# # Merge data
# - name: Run Script merge_data.py
# if: ${{ github.event.inputs.commitToRepos == 'true' || github.event.schedule == '0 12 * * FRI' }}
# run: python ./merge_data.py
# working-directory: ${{env.odbodsPath}}

# # Convert merged CSV to Markdown files
# - name: Run Script export2jkan.py
# if: ${{ github.event.inputs.commitToRepos == 'true' || github.event.schedule == '0 12 * * FRI' }}
# run: python ./export2jkan.py
# working-directory: ${{env.odbodsPath}}

# Run pipeline
- name: Build and run docker container
run: |
docker build -t open_data_scotland_pipeline .
docker run -t --name ods_pipeline open_data_scotland_pipeline
working-directory: ${{env.odbodsPath}}

# Extract files from Docker container
- name: Copy container contents and overwrite datasets
run: |
rm -r jkan/_datasets/*
docker cp ods_pipeline:./usr/src/app/. .
- name: Check files
run: ls -l -a

# Commit changes
- name: Git config
working-directory: ${{env.jkanPath}}

0 comments on commit bbcc4c8

Please sign in to comment.