FATAL: Database migration failed

[shiipou]

Type

Crash

Instances

• Canary
• Stable
• Self-Hosted

Description

We can't run the project because of a database migration error.

Steps to reproduce

1. Clone the repo with git clone git@github.com:zekroTJA/shinpuru
2. Go to the shinpuru directory with cd shinpuru
3. Start databases and other services with docker-compose -f .\docker-compose.dev.yml up
4. Open vscode in that directory with code .
5. Run the project by pressing F5 key.

Attachments

My configuration file (I've removed tokens and secrets) :

# This just marks the version of the config.
# This is essential for figuring out if the config
# structure has changed with an update and if the config
# will be not validly parsed.
configVersionPleaseDoNotChange: 6

# Discord specific configuration
discord:
  # Your bot's token you will get from https://discord.com/developers/applications
  token: ""
  # The global prefix which will be always usable on all guilds
  generalprefix: "sp!"
  # The Discord ID of the host of the bot
  # This user will have FULL PERMISSIONS on all guilds
  ownerid: ""
  # The client ID of the bot's application account
  # Get this from https://discord.com/developers/applications
  clientid: ""
  # The client secret for the bot's application account
  # Get this from https://discord.com/developers/applications
  clientsecret: ""
  # Specify properties for the global command
  # rate limiter middleware.
  globalcommandratelimit:
    # Rate limit bucket token size.
    burst: 1
    # Rate limit bucket token regeneration
    # in seconds.
    limitseconds: 10

# Default permissions for users and admins
permissions:
  # Default permissions for users without any
  # previleges
  defaultuserrules:
  - +sp.etc.*
  - +sp.chat.*
  # Default permissions for guild owners
  # and admins
  defaultadminrules:
  - +sp.guild.*
  - +sp.etc.*
  - +sp.chat.*

# Database sepcific configuration
database:
  # Currently, this bot supports only MySql (MariaDB).
  # But when more database bindings are available, here
  # you can choose which one to use.
  type: "mysql"
  # MySQL (MariaDB) configuration
  mysql:
    # Host address of the database
    host: "localhost:3306"
    # Username of the database account
    user: "root"
    # Password for the used database account
    password: "dev"
    # Database name
    database: "shinpuru"

# Caching prefrences.
cache:
  # Redis connection configuration.
  redis:
    # Redis host address
    addr: "localhost:6379"
    # Redis password
    password: "myredispassword"
    # Database type
    type: 0
  # If enabled, most frequently used database
  # requests are automatically cached in redis
  # to minimize load on the database as well as
  # request times.
  # It is recomendet to leave this enabled. If
  # you want to disable it for whatever reason,
  # you can do it here.
  cachedatabase: false

# Logging preferences
logging:
  # Print log output each time a command is called
  commandlogging: true
  # Set the log level of the logger
  # Log levels can be found here:
  # https://github.com/op/go-logging/blob/master/level.go#L20
  loglevel: 4

# Storage interface configuration
storage:
  # Here you can chose between 'file' or
  # 'minio' as storage interfaces.
  # Minio also supprts connecting to Amazon S3
  # or google Cloud.
  type: "minio"
  # Minio interface configuration
  minio:
    # Endpoint address of the storage
    endpoint: "localhost:9000"
    # Storage access key
    accesskey: "dev-access-key"
    # Storage access secret
    accesssecret: "dev-secret-key"
    # Main location to be used
    location: "us-east-1"
    # Use HTTPS for API calls
    secure: true
  # File interface configuration
  file:
    # Local location of the data storage
    location: ./data

# Web server configuration
webserver:
  # Whether to expose web server and REST API
  enabled: true
  # Address to expose web server to
  addr: 0.0.0.0:8080
  # TLS configuration
  tls:
    # Use TLS (HTTPS)
    enabled: false
    # TLS certificate location
    cert: "/etc/cert/mycert.pem"
    # TLS keyfile location
    key: "/etc/cert/mykey.pem"
  # The address, the web server will be accessable
  # This value MUST be the same address as configured
  # in the OAuth2 preferences of your bots application
  publicaddr: "https://dev.nocturlab.fr"
  # Debug address which is set during debugging as
  # CORS origin domain.
  debugpublicaddr: "http://localhost:8081"
  # Ratelimit configuration
  ratelimit:
    # Whether or not to enable rate limiter
    enabled: true
    # Burst defines max requests in the given duration.
    # This value should not be that low, because first
    # connections to the API via the web interface might
    # require a lot of requests to be processed.
    burst: 50
    # The reset duration until a request token is restored.
    limitseconds: 3

# Credentials of the twitch app to connect to the
# twitch API
# You can register a twitch API app and get the
# credentials from this page:
# https://glass.twitch.tv/console/apps
twitchapp:
  # Client ID
  clientid: ""
  # Client Secret
  clientsecret: ""

# Prometheus metrics endpoint configuration
metrics:
  # Whether or not to enable the prometheus
  # metrics and HTTP endpoint
  enable: true
  # The address the metrics HTTP server binds
  # to. This is defaultly ":9091" if not
  # specified
  addr: ":9091"

# Time Schedule specifications.
# Time schedules are specified using the crontab
# syntax:
#   second minute hour month dom dow
# Read this to get more information:
# https://pkg.go.dev/github.com/robfig/cron/v3#hdr-Usage
schedules:
  # Guild backup schedule
  guildbackups:        '0 0 6,18 * * *'
  # Refresh token cleanup schedule
  refreshtokencleanup: '0 0 5 * * *'

Some logs :

Starting: C:\Users\flavi\go\bin\dlv-dap.exe dap --check-go-version=false --listen=127.0.0.1:50450 from C:\Users\flavi\Documents\dev\github.com\shiipou\shinpuru/cmd/shinpuru
DAP server listening at: 127.0.0.1:50450
Type 'dlv help' for list of commands.
------------------------------------------------
                                                
        _     _                                 
       | |   (_)                                
    ___| |__  _ _ __  _ __  _   _ _ __ _   _    
   / __| '_ \| | '_ \| '_ \| | | | '__| | | |   
   \__ \ | | | | | | | |_) | |_| | |  | |_| |   
   |___/_| |_|_|_| |_| .__/ \__,_|_|   \__,_|   
                    | |                         
                    |_|                         
                                                
   shinpuru
   © 2021 Ringo Hoffmann (zekro Development)
   
   Version: TESTING_BUILD
   Commit:  TESTING_BUILD
   Release: false

------------------------------------------------
INFO[2021/12/31 12:23:32 CET] Starting up...
INFO[2021/12/31 12:23:33 CET] Checking database for migrations and apply if needed...
INFO[2021/12/31 12:23:33 CET] DATABASE :: applying migration                version=0
INFO[2021/12/31 12:23:33 CET] DATABASE :: applying migration                version=1
INFO[2021/12/31 12:23:33 CET] DATABASE :: applying migration                version=2
INFO[2021/12/31 12:23:33 CET] DATABASE :: applying migration                version=3
INFO[2021/12/31 12:23:33 CET] DATABASE :: applying migration                version=4
INFO[2021/12/31 12:23:33 CET] DATABASE :: applying migration                version=5
INFO[2021/12/31 12:23:33 CET] DATABASE :: applying migration                version=6
FATA[2021/12/31 12:23:33 CET] Database migration failed                     error="Error 1075: Incorrect table definition; there can be only one auto column and it must be defined as a key"
Process 21904 has exited with status 1
Detaching
dlv dap (6384) exited with code: 0

shiipou\shinpuru>docker-compose -f .\docker-compose.dev.yml up
[+] Running 7/7
 - Volume "shinpuru_minio"          Created             0.0s 
 - Volume "shinpuru_mysql-cfg"      Created             0.0s 
 - Volume "shinpuru_mysql-lib"      Created             0.0s 
 - Container shinpuru_redis_1       Created             0.1s 
 - Container shinpuru_mysql_1       Created             0.1s 
 - Container shinpuru_minio_1       Created             0.1s 
 - Container shinpuru_phpmyadmin_1  Created             0.1s
Attaching to minio_1, mysql_1, phpmyadmin_1, redis_1
redis_1       | 1:C 31 Dec 2021 11:23:19.563 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis_1       | 1:C 31 Dec 2021 11:23:19.563 # Redis version=6.2.6, bits=64, commit=00000000, modified=0, pid=1, just started
redis_1       | 1:C 31 Dec 2021 11:23:19.563 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf     
redis_1       | 1:M 31 Dec 2021 11:23:19.564 * monotonic clock: POSIX clock_gettime
redis_1       | 1:M 31 Dec 2021 11:23:19.564 * Running mode=standalone, port=6379.
redis_1       | 1:M 31 Dec 2021 11:23:19.564 # Server initialized
redis_1       | 1:M 31 Dec 2021 11:23:19.564 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.      
redis_1       | 1:M 31 Dec 2021 11:23:19.564 * Ready to accept connections
mysql_1       | 2021-12-31 11:23:19+00:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:10.6.5+maria~focal started.
mysql_1       | 2021-12-31 11:23:19+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
mysql_1       | 2021-12-31 11:23:19+00:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:10.6.5+maria~focal started.
mysql_1       | 2021-12-31 11:23:20+00:00 [Note] [Entrypoint]: Initializing database files
minio_1       | WARNING: MINIO_ACCESS_KEY and MINIO_SECRET_KEY are deprecated.
minio_1       |          Please use MINIO_ROOT_USER and MINIO_ROOT_PASSWORD
minio_1       | API: http://172.18.0.2:9000  http://127.0.0.1:9000
minio_1       |
minio_1       | Console: http://172.18.0.2:36539 http://127.0.0.1:36539
minio_1       |
minio_1       | Documentation: https://docs.min.io
minio_1       |
minio_1       | WARNING: Console endpoint is listening on a dynamic port (36539), please use --console-address ":PORT" to choose a static port.
mysql_1       | 2021-12-31 11:23:20 0 [Warning] You need to use --log-bin to make --expire-logs-days or --binlog-expire-logs-seconds work.
phpmyadmin_1  | AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.18.0.5. Set the 'ServerName' directive globally to suppress this message
phpmyadmin_1  | AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.18.0.5. Set the 'ServerName' directive globally to suppress this message
phpmyadmin_1  | [Fri Dec 31 11:23:20.865332 2021] [mpm_prefork:notice] [pid 1] AH00163: Apache/2.4.38 (Debian) PHP/7.4.20 configured -- resuming normal operations
phpmyadmin_1  | [Fri Dec 31 11:23:20.865370 2021] [core:notice] [pid 1] AH00094: Command line: 'apache2 -D FOREGROUND'    
mysql_1       | 
mysql_1       | 
mysql_1       | PLEASE REMEMBER TO SET A PASSWORD FOR THE MariaDB root USER !
mysql_1       | To do so, start the server, then issue the following command:
mysql_1       |
mysql_1       | '/usr/bin/mysql_secure_installation'
mysql_1       |
mysql_1       | which will also give you the option of removing the test
mysql_1       | databases and anonymous user created by default.  This is
mysql_1       | strongly recommended for production servers. 
mysql_1       |
mysql_1       | See the MariaDB Knowledgebase at https://mariadb.com/kb or the
mysql_1       | MySQL manual for more instructions.
mysql_1       |
mysql_1       | Please report any problems at https://mariadb.org/jira
mysql_1       |
mysql_1       | The latest information about MariaDB is available at https://mariadb.org/.
mysql_1       | You can find additional information about the MySQL part at:
mysql_1       | https://dev.mysql.com
mysql_1       | Consider joining MariaDB's strong and vibrant community:
mysql_1       | https://mariadb.org/get-involved/
mysql_1       |
mysql_1       | 2021-12-31 11:23:23+00:00 [Note] [Entrypoint]: Database files initialized
mysql_1       | 2021-12-31 11:23:23+00:00 [Note] [Entrypoint]: Starting temporary server
mysql_1       | 2021-12-31 11:23:23+00:00 [Note] [Entrypoint]: Waiting for server startup
mysql_1       | 2021-12-31 11:23:23 0 [Note] mariadbd (server 10.6.5-MariaDB-1:10.6.5+maria~focal) starting as process 127 ...
mysql_1       | 2021-12-31 11:23:23 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
mysql_1       | 2021-12-31 11:23:23 0 [Note] InnoDB: Number of pools: 1
mysql_1       | 2021-12-31 11:23:23 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
mysql_1       | 2021-12-31 11:23:23 0 [Note] mariadbd: O_TMPFILE is not supported on /tmp (disabling future attempts)     
mysql_1       | 2021-12-31 11:23:23 0 [Note] InnoDB: Using Linux native AIO
mysql_1       | 2021-12-31 11:23:23 0 [Note] InnoDB: Initializing buffer pool, total size = 134217728, chunk size = 134217728
mysql_1       | 2021-12-31 11:23:23 0 [Note] InnoDB: Completed initialization of buffer pool
mysql_1       | 2021-12-31 11:23:23 0 [Note] InnoDB: 128 rollback segments are active.
mysql_1       | 2021-12-31 11:23:23 0 [Note] InnoDB: Creating shared tablespace for temporary tables
mysql_1       | 2021-12-31 11:23:23 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
mysql_1       | 2021-12-31 11:23:23 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
mysql_1       | 2021-12-31 11:23:23 0 [Note] InnoDB: 10.6.5 started; log sequence number 42161; transaction id 14
mysql_1       | 2021-12-31 11:23:23 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
mysql_1       | 2021-12-31 11:23:23 0 [Note] Plugin 'FEEDBACK' is disabled.
mysql_1       | 2021-12-31 11:23:23 0 [Note] InnoDB: Buffer pool(s) load completed at 211231 11:23:23
mysql_1       | 2021-12-31 11:23:23 0 [Warning] You need to use --log-bin to make --expire-logs-days or --binlog-expire-logs-seconds work.
mysql_1       | 2021-12-31 11:23:23 0 [Warning] 'user' entry 'root@0f5bfd1222f8' ignored in --skip-name-resolve mode.     
mysql_1       | 2021-12-31 11:23:23 0 [Warning] 'proxies_priv' entry '@% root@0f5bfd1222f8' ignored in --skip-name-resolve mode.
mysql_1       | 2021-12-31 11:23:23 0 [Note] mariadbd: ready for connections.
mysql_1       | Version: '10.6.5-MariaDB-1:10.6.5+maria~focal'  socket: '/run/mysqld/mysqld.sock'  port: 0  mariadb.org binary distribution
mysql_1       | 2021-12-31 11:23:24+00:00 [Note] [Entrypoint]: Temporary server started.
mysql_1       | Warning: Unable to load '/usr/share/zoneinfo/leap-seconds.list' as time zone. Skipping it.
mysql_1       | Warning: Unable to load '/usr/share/zoneinfo/leapseconds' as time zone. Skipping it.
mysql_1       | Warning: Unable to load '/usr/share/zoneinfo/tzdata.zi' as time zone. Skipping it.
mysql_1       | 2021-12-31 11:23:26 5 [Warning] 'proxies_priv' entry '@% root@0f5bfd1222f8' ignored in --skip-name-resolve mode.
mysql_1       | 2021-12-31 11:23:26+00:00 [Note] [Entrypoint]: Creating database shinpuru
mysql_1       |
mysql_1       | 2021-12-31 11:23:26+00:00 [Note] [Entrypoint]: Stopping temporary server
mysql_1       | 2021-12-31 11:23:26 0 [Note] mariadbd (initiated by: root[root] @ localhost []): Normal shutdown
mysql_1       | 2021-12-31 11:23:26 0 [Note] InnoDB: FTS optimize thread exiting.
mysql_1       | 2021-12-31 11:23:26 0 [Note] InnoDB: Starting shutdown...
mysql_1       | 2021-12-31 11:23:26 0 [Note] InnoDB: Dumping buffer pool(s) to /var/lib/mysql/ib_buffer_pool
mysql_1       | 2021-12-31 11:23:26 0 [Note] InnoDB: Buffer pool(s) dump completed at 211231 11:23:26
mysql_1       | 2021-12-31 11:23:26 0 [Note] InnoDB: Removed temporary tablespace data file: "./ibtmp1"
mysql_1       | 2021-12-31 11:23:26 0 [Note] InnoDB: Shutdown completed; log sequence number 42173; transaction id 15     
mysql_1       | 2021-12-31 11:23:26 0 [Note] mariadbd: Shutdown complete
mysql_1       |
mysql_1       | 2021-12-31 11:23:27+00:00 [Note] [Entrypoint]: Temporary server stopped
mysql_1       | 
mysql_1       | 2021-12-31 11:23:27+00:00 [Note] [Entrypoint]: MariaDB init process done. Ready for start up.
mysql_1       |
mysql_1       | 2021-12-31 11:23:27 0 [Note] mariadbd (server 10.6.5-MariaDB-1:10.6.5+maria~focal) starting as process 1 ...
mysql_1       | 2021-12-31 11:23:27 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
mysql_1       | 2021-12-31 11:23:27 0 [Note] InnoDB: Number of pools: 1
mysql_1       | 2021-12-31 11:23:27 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
mysql_1       | 2021-12-31 11:23:27 0 [Note] mariadbd: O_TMPFILE is not supported on /tmp (disabling future attempts)     
mysql_1       | 2021-12-31 11:23:27 0 [Note] InnoDB: Using Linux native AIO
mysql_1       | 2021-12-31 11:23:27 0 [Note] InnoDB: Initializing buffer pool, total size = 134217728, chunk size = 134217728
mysql_1       | 2021-12-31 11:23:27 0 [Note] InnoDB: Completed initialization of buffer pool
mysql_1       | 2021-12-31 11:23:27 0 [Note] InnoDB: 128 rollback segments are active.
mysql_1       | 2021-12-31 11:23:27 0 [Note] InnoDB: Creating shared tablespace for temporary tables
mysql_1       | 2021-12-31 11:23:27 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
mysql_1       | 2021-12-31 11:23:27 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
mysql_1       | 2021-12-31 11:23:27 0 [Note] InnoDB: 10.6.5 started; log sequence number 42173; transaction id 14
mysql_1       | 2021-12-31 11:23:27 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
mysql_1       | 2021-12-31 11:23:27 0 [Note] Plugin 'FEEDBACK' is disabled.
mysql_1       | 2021-12-31 11:23:27 0 [Note] InnoDB: Buffer pool(s) load completed at 211231 11:23:27
mysql_1       | 2021-12-31 11:23:27 0 [Warning] You need to use --log-bin to make --expire-logs-days or --binlog-expire-logs-seconds work.
mysql_1       | 2021-12-31 11:23:27 0 [Note] Server socket created on IP: '0.0.0.0'.
mysql_1       | 2021-12-31 11:23:27 0 [Note] Server socket created on IP: '::'.
mysql_1       | 2021-12-31 11:23:27 0 [Warning] 'proxies_priv' entry '@% root@0f5bfd1222f8' ignored in --skip-name-resolve mode.
mysql_1       | 2021-12-31 11:23:27 0 [Note] mariadbd: ready for connections.
mysql_1       | Version: '10.6.5-MariaDB-1:10.6.5+maria~focal'  socket: '/run/mysqld/mysqld.sock'  port: 3306  mariadb.org binary distribution
mysql_1       | 2021-12-31 11:23:33 3 [Warning] Aborted connection 3 to db: 'shinpuru' user: 'root' host: '172.18.0.1' (Got an error reading communication packets)

[shiipou]

Here the screenshot of the database migration and table list :

[shiipou]

Migration 6 seems to be properly patched but throw an error at launch :

So I can workaround this error just by adding the migration 6 manually on the database.

Now I got a new error :

panic: could not build `kencommandhandler`: ERR AUTH <password> called without any password configured for the default user. Are you sure your configuration is correct?

goroutine 1 [running]:
github.com/sarulabs/di/v2.(*containerGetter).Get(0x0, 0xc000342550, {0x157d90b, 0x11})
	C:/Users/flavi/go/pkg/mod/github.com/sarulabs/di/v2@v2.4.2/containerGetter.go:17 +0x11e
github.com/sarulabs/di/v2.(*container).Get(0xc000342550, {0x157d90b, 0x11})
	C:/Users/flavi/go/pkg/mod/github.com/sarulabs/di/v2@v2.4.2/container.go:38 +0x4d
main.main()
	C:/Users/flavi/Documents/dev/github.com/shiipou/shinpuru/cmd/shinpuru/main.go:361 +0x1ae8
Process 19880 has exited with status 2
Detaching
dlv dap (19636) exited with code: 0

[zekroTJA]

Hey, thank you for the report.

I can in fact reproduce the issue. I've never tried the migrations on a fresh database since a long time. I will definitely ensure that the fresh setup works the next time before releasing a new version.

I will look into it and provide a fix asap. ^^