-
Notifications
You must be signed in to change notification settings - Fork 4.8k
Harbor FAQs
-
When install Harbor it report follow Error:
Traceback (most recent call last): File "./prepare", line 110, in <module> validate(rcp, args) File "./prepare", line 31, in validate raise Exception("Error: The path for certificate: %s is invalid" % cert_path) Exception: Error: The path for certificate: /data/cert/server.crt is invalid
Root cause: This error was caused by when you enable https on harbor.cfg but don't have "/data" directory and create the harbor server ca certificate and server certificateSolution: Make sure you have a /data directory and follow the step in [enable https] (https://github.com/vmware/harbor/blob/master/docs/configure_https.md)
-
How to customize the port that Harbor listens on?
[A] Please refer to the [installation guide] (https://github.com/vmware/harbor/blob/master/docs/installation_guide.md#configuring-harbor-listening-on-a- customized-port).
-
How to initialized Harbor DB when use external database.
[A] Please refer [load-harbor-db-schema] (https://github.com/vmware/harbor/blob/master/docs/high_availability_installation_guide.md#load-harbor-db- schema)
-
Can't find the download certificate button
[A] Copy the ca certificate to /data/ca_download/ca.crt, then this download link is visible in web console.
-
Internal certs enabled Harbor fail to start up after upgrading from the version before v2.2.0 to newer version.
[A] This may caused by Golang version upgrading to
1.5.0
. Which force the SAN extension included in cert file. So you need to add SAN to your certs file. More information please refer this
Postgres is the database used by the harbor, the default max_connections
of the database is 100. This value is small sometimes and may cause "pq: sorry, too many clients already" issue. The following is the workaround to modify the max_connections
to 1000 manually.
- Goto the directory of the installer
- Modify the
max_connections
parameter in /data/database/postgresql.conf
max_connections=1000
- Restart the postgresql
sudo docker-compose restart postgresql
- Ensure that the
max_connections
changed to 1000
sudo docker-compose exec postgresql psql -c "SHOW max_connections"
max_connections
-----------------
1000
(1 row)
- Fetch the
postgresql.conf
from the database pod
NAMESPACE=the-namespace-of-harbor
POD_NAME=`kubectl -n $NAMESPACE get pod -l "component=database,app=harbor" -o name`
kubectl -n $NAMESPACE exec $POD_NAME -- cat /var/lib/postgresql/data/postgresql.conf > postgresql.conf
- Modify the
max_connections
parameter in the postgresql.conf
max_connections=1000
- Write back the
postgresql.conf
to the database
NAMESPACE=the-namespace-of-harbor
POD_NAME=`kubectl -n $NAMESPACE get pod -l "component=database,app=harbor" -o name`
cat postgresql.conf | kubectl -n $NAMESPACE exec $POD_NAME -i -- tee /var/lib/postgresql/data/postgresql.conf
- Delete the pod of the database and wait Kubernetes schedule a new pod
kubectl -n $NAMESPACE delete $POD_NAME
kubectl -n $NAMESPACE wait --for=condition=ready $POD_NAME --timeout=60s
- Ensure that the
max_connections
changed to 1000
kubectl -n $NAMESPACE exec $POD_NAME -- psql -c "SHOW max_connections"
max_connections
-----------------
1000
(1 row)
Set an environment for harbor-db
POSTGRES_MAX_CONNECTIONS = 1000
Then restart harbor-db container or pod, it will pickup the configuration from env.
-
What happens if I update a same name image to Harbor with replication enabled?
[A] It will overwrite the images both on the source and destination Harbor server
-
Got
504 Gateway Time-out
error when replicating big images.[A] Please refer to issue 3446.
-
Can harbor send webhook notification like https://docs.docker.com/docker-hub/webhooks/ showing?
[A] No, Harbor does not support webhook notification of registry yet.
-
If there are some replication tasks are
hung
(status is hung in the pending/running status), how can I fix them?[A] Several ways you can have a try:
- Via harbor portal: Login Harbor as an administrator, select the numeric ID of the replication execution which has hung tasks under the
replication
section. Click theStop
button to stop the tasks. - If the above approach can not resolve the "hung" issue, you can connect to the database to directly update the status of the hung tasks.
# Make sure harbor is still running docker exec -it harbor-db /bin/bash # Enter the db container # Connect to the database psql -U postgres # Change database to registry \c registry # Update the database records UPDATE replication_task SET status = 'Stopped' WHERE status = 'InProgress';
- Via harbor portal: Login Harbor as an administrator, select the numeric ID of the replication execution which has hung tasks under the
-
Can I use the scan functionality when Harbor has no internet access?
[A] You can but you need to manually update the vulnerability database. please refer to this wiki: [Import Clair vulnerability data] (https://github.com/vmware/harbor/blob/master/docs/import_vulnerability_data.md)
-
Scan all has been issued for a long while, but there still are some scanning tasks not completed. How can I fix it?
[A] If the "Scan All" button is not disabled, you can click again. The new scanning processes will override the previous ones. If the "Scan All" button is disabled, you can also trigger a new scan all process by issuing an API call.
curl -X POST -u <USER>:<PASSWORD> -H "Content-type: application/json" -H "X-Xsrftoken:xtuwrDBPMSbkNR0r7rchHdpjX57o26By" -k -i -d '{"schedule":{"type":"Manual"}}' https://<HOST>/api/system/scanAll/schedule
-
Can I disable vulnerability scan against language-specific package and/or associated library packages?
[A] If users deploy a harbor instance via harbor-helm, they can set trivy.vulnType to
os
only, instead of the default value ofos,library
, so that the Language-specific packages or associated library packages will not be scanned for vulnerabilities.
-
Why can not push image 192.168.0.1/hello-world:latest to Harbor?
[A] At least two namespaces are needed for repository name in Harbor, so tag the image as 192.168.0.1/project_name/hello-world:latest should fix this. (Create the project on the web page first)
-
How to access the APIs of the Docker registry?
[A] First you need to request a token:curl -i -k -u <username>:<password> https://<harbor_host_or_ip>/service/token?service=harbor- registry&scope=repository:library/mysql:pull,push
Then you can use the token to issue registry API:
curl -i -k -H "Content-Type: application/json" -H "Authorization: Bearer longlongtokenxxxxx“ -X GET https://10.192.212.107/v2/library/mysql/5.6.35/manifests/latest
About the detail of the token, please refer to the guide https://github.com/docker/distribution/blob/master/docs/spec/auth/token.md.
-
How to issue an API call in Harbor with version >= 1.10?
[A] XSRF protection is enabled in Harbor V1.10. After this version, when you issue an API call, you can
# Send API call
curl -X POST -u <USER>:<PASSWORD> -H "Content-type: application/json" -k -i -d '{"data": "example"}' https://<HOST>/api/system/scanAll/schedule
# If 403 code is returned, extract the "_xsrf" value from the "Set-Cookie" header
# e.g: Set-Cookie: _xsrf=eHR1d3JEQlBNU2JrTlIwcjdyY2hIZHBqWDU3bzI2Qnk=|1576117983208824582|c861b53e3e4fdcfc2f8f5b6c38cc14ed0a1272a8; Expires=Thu, 12 Dec 2019 03:33:03 UTC; Max-Age=3600; Path=/; Secure
# As the _xsrf value is encoded with base64, decode it with base64 and get the raw value
# e.g: base64-decode "eHR1d3JEQlBNU2JrTlIwcjdyY2hIZHBqWDU3bzI2Qnk=" to xtuwrDBPMSbkNR0r7rchHdpjX57o26By
# Add "X-Xsrftoken" header with the decoded value and issue the API call again
curl -X POST -u <USER>:<PASSWORD> -H "Content-type: application/json" -H "X-Xsrftoken:xtuwrDBPMSbkNR0r7rchHdpjX57o26By" -k -i -d '{"data": "example"}' https://<HOST>/api/system/scanAll/schedule
- My Harbor is configured to use OIDC for authentication, how do I access Harbor's API?
[A] In such case, you have to use the OIDC token as bearer token to access Harbor's API.
Keycloak example, please refer to:
https://github.com/goharbor/harbor/issues/10597#issuecomment-603159112
Azure AD example, please refer to:
https://github.com/goharbor/harbor/issues/9193#issuecomment-1317557916
Dex example, please refer to:
https://dexidp.io/docs/using-dex/
Or set the Harbor log level to debug and get the bearer token from the core.log
Raw ID token for verification: <bearer token>
-
How to change auth mode when the auth_mode is not editable?
[A] Execute the following command to make the auth_mode editable
docker exec -it harbor-db bash psql -U postgres \c registry select * from harbor_user; delete from harbor_user where user_id > 2;
Refresh the Harbor web console-> Configurations, then you can change the auth_mode.
-
How to reset admin password?
[A] The initial admin password can be found in harbor.yml,
harbor_admin_password: <initial_admin_password>
If the administrator have update his password in web console and forget it.
Make sure the harbor server is running.
docker exec -it harbor-db psql -U postgres \c registry update harbor_user set salt='', password='' where user_id = 1;
Restart Harbor
docker-compose down -v docker-compose up -d
Then you can login with the initial admin password
- If the Harbor is installed in Kubernetes, you could reset the password with the same approache, just replace the docker command with kubectl command and restart the harbor-core pod after updating the harbor_user table.
-
How to configure Azure Active Directory as the OIDC provider?
[A] Refer to this issue: https://github.com/goharbor/harbor/issues/9193#issuecomment-1317557916
-
When auth mode is changed to ldap_auth, all LDAP users can login harbor, How to let only users in a group can login?
[A] You can add LDAP filter like that:
(&(objectclass=person)(memberof=CN=harbor_users,OU=sample,OU=vmware,DC=harbor,DC=com))
The
CN=harbor_users,OU=sample,OU=vmware,DC=harbor,DC=com
is the LDAP group DN, then only LDAP user in group harbor_users can login. -
When LDAP UID setting is changed, some LDAP user can not login
[A] Because the LDAP user have logged in with different UID, some user information is cached in the Harbor DB. you can clean up the user information and try login again.
docker exec -it harbor-db bash psql -U postgres \c registry select * from harbor_user; delete from harbor_user where user_id > 2;
-
How to add a CA cert for the LDAP server or other Harbor server?
[A] After installed the Harbor, there is a directory under common/config/shared/trust-certificates Copy the LDAP certificate, for example, ldap_ca.crt to this directory and restart the Harbor, The certificate is added to the trust store of the core container, and then you could enable “Verify Cert” in the LDAP configuration.
-
How can I workaround the CVE-2019-16097?
[A] The system admin can disable the
allow self-registration
both via UI or API.- UI: Configuration -> Authentication -> Allow Self-Registration(uncheck the checkbox)
- API: Use the configuration API to update Self-Registration.
PUT /api/configurations {"self_registration":false}
Before the Harbor V2.1 release, the Harbor will be put into read-only
mode when the GC task is started and change back to normal mode when GC is completed. GC task can also be set as a scheduled task that is executed periodically. Sometimes the GC task that is running in jobservice may be failed to report the status change to the Harbor core and then cause the read-only
mode can not be changed back. The whole harbor is stuck in the read-only
mode as the GC task is periodically running.
- A quick workaround solution
Disable the
read-only
mode in the system configuration page immediately.
-
Workaround solution at jobservice side
-
Enter your Redis container
docker exec -ti redis /bin/bash
-
Connect to the Redis
redis-cli -n 2
2 is the db index for jobservice (you can find the settings in the installation configuration yaml fileharbror.yaml
) -
Find all the schedule policies related to
IMAGE_GC
wth commandzrange {harbor_job_service_namespace}:period:policies 0 -1
- Delete them with the command
ZREMRANGEBYLEX {harbor_job_service_namespace}:period:policies [INDEX] [INDEX]
NOTE: [INDEX] is the ID number you found at step3
- Delete them with the command
-
Find all the tasks that have been scheduled for executing and related to
IMAGE_GC
with commandzrange {harbor_job_service_namespace}:scheduled 0 -1
- Delete them with the command
ZREMRANGEBYLEX {harbor_job_service_namespace}:scheduled [INDEX] [INDEX]
NOTE: [INDEX] is the ID number you found at step4
- Delete them with the command
-
Reset your GC schedule at the GC management page if necessary
If you're experiencing the key not found error descripted as https://github.com/goharbor/harbor/issues/14932 in the particular path mentioned by https://github.com/goharbor/harbor/commit/4017e995b7ada3bcb54ba30b2a86a7559cbcf1a9. Please follow the steps to resolve:
- Remove gun from notary DB.
docker exec -it harbor-db bash
psql -U postgres
\c notarysigner
delete from private_keys where gun = '${host}/${project}/${repository}';
\c notaryserver
delete from tuf_files where gun = '${host}/${project}/${repository}';
- Remove local tuf
rm -rf ${notary_cache_directory}/tuf/${host}/${project}/${repository}/*
By default, notary is using ~/.docker/trust
as the local cache directory.
The trivy scanner can be used in air-gapped environments. The following is the steps.
- Install the harbor with
skip_update=true
for the trivy configuration. - Download the vulnerability database
$ wget https://github.com/aquasecurity/trivy-db/releases/latest/download/trivy-offline.db.tgz
$ tar xvf trivy-offline.db.tgz
x trivy.db
x metadata.json
$ chmod o+r trivy.db
$ chmod o+r metadata.json
- Put the DB file in Trivy's cache directory
For the harbor installed by the installer
$ docker exec -u scanner trivy-adapter mkdir -p /home/scanner/.cache/trivy/db/
$ docker cp metadata.json trivy-adapter:/tmp/metadata.json
$ docker cp trivy_db/trivy.db trivy-adapter:/tmp/trivy.db
$ docker exec -u scanner trivy-adapter cp /tmp/metadata.json /home/scanner/.cache/trivy/db/metadata.json
$ docker exec -u scanner trivy-adapter cp /tmp/trivy.db /home/scanner/.cache/trivy/db/trivy.db
For the harbor installed by the harbor-helm
$ NAMESPACE=the-namespace-of-harbor
$ POD_NAME=`kubectl -n $NAMESPACE get pod -l "component=trivy" -o name`
$ kubectl -n $NAMESPACE exec $POD_NAME -- mkdir -p /home/scanner/.cache/trivy/db/
$ cat metadata.json | kubectl -n $NAMESPACE exec $POD_NAME -i -- tee /home/scanner/.cache/trivy/db/metadata.json 1>/dev/null
$ cat trivy.db | kubectl -n $NAMESPACE exec $POD_NAME -i -- tee /home/scanner/.cache/trivy/db/trivy.db 1>/dev/null
- Failed to delete the existing chart
When deleting an existing chart, failed with error: "fail to get chart version: improper constraint: xxx-xxx-xxx" This issue often happens when the chart version doesn't follow the semver, it can be uploaded, but can't be removed. The workaround to remove these charts
# Stop the Harbor
cd /data/chart_storage/<project of the chartmuseum uploaded>
# remove chart files
rm <chart files>.tgz
# Remove the index-cache.yaml, it will generate when next start
rm -rf /data/chart_storage/<project of the chartmuseum uploaded>/index-cache.yaml
# cleanup redis cache
rm /data/redis/*
# Start the Harbor
# Then deleted charts should be removed from the Harbor, remain charts should be visible.
- Failed to edit System Robot Account with all-projects covering.
A regression issue was introduced in Harbor v2.5.3 which will cause the failure of updating the System Robot Account with all-projects access. You can use the bellowing workaround to recreate your System Robot Account without usage impact.
1. You need to know the name and secret of the System Robot Account you want to edit.
2. Delete this System Robot Account.
3. Create a new System Robot Account and enter your previous System Robot Account name and the fields(Expiration time, Description) you want to edit.
4. Refresh the secret of this System Robot Account with previous secret.
5. This will not affect your System Robot Account usage because the name and secret are not changed.
-- task
CREATE INDEX IF NOT EXISTS idx_task_job_id ON task (job_id);
-- execution
CREATE INDEX IF NOT EXISTS idx_execution_vendor_type_vendor_id ON execution (vendor_type, vendor_id)
CREATE INDEX IF NOT EXISTS idx_execution_start_time ON execution(start_time)
-- audit_log
CREATE INDEX IF NOT EXISTS idx_audit_log_project_id ON audit_log (project_id);
-- artifact
-- CREATE INDEX IF NOT EXISTS idx_artifact_repository_name ON artifact (repository_name);
-- or
CREATE INDEX concurrently idx_artifact_repository_name ON artifact USING hash (repository_name);
CREATE INDEX IF NOT EXISTS idx_artifact_repository_id ON artifact (repository_id);
CREATE INDEX IF NOT EXISTS idx_artifact_project_id ON artifact (project_id);
-- remove index has drawback
DROP INDEX IF EXISTS idx_audit_log_op_time;
- There are too many jobs in pending/running state, how to clean it?
1. Stop all the jobs with a schedule, such as tag retention, image scan and GC job.
2. Mark pending task into Error
> docker exec -it harbor-db bash
> psql -U postgres -d registry
> update task set status = 'Error', status_code = 3 where status in ('Pending', 'Running') and vendor_type = '<job_type>';
> update execution set status = 'Stopped' where status = 'Running' and vendor_type = '<job type>';
3. Backup the redis file to another directory and delete it, then restart Harbor
> docker-compose down -v
> cp -r /data/redis/* /data/redis_backup/
> rm -rf /data/redis/*
> docker-compose up -d
Because harbor proxy relies on HEAD request to link the digest and the tag. If the tag and its digest information are cached locally, then it won't send the HEAD request to the server, then the Harbor doesn't link the tag and the digest.
Sometimes when deploy application in kubernetes, why the repository's pull count is not updated?
[A] It is the implementation of the container runtime, first it sends a HEAD request to the registry, and then the client gets the digest of the manifest from the response, and if there is any image in the local cache with the same digest, it uses the image in the local cache and skip to pull the image, that is why the pull count is not updated. Only GET requests to manifest update the pull count, HEAD requests have no impact on the pull count.