Tidy up docs and remove volumes code; use /complement (#304)

- Base mounted files under `/complement`
- Remove `Volume` code, we don't need it as we have `copyToContainer` now.
- Remove Dendrite dockerfiles and point to the complement dockerfile in the dendrite repo.
- Remove Complement CI buildkit dockerfile, we don't run complement in buildkite anymore.
- Fix up README/ONBOARDING to reflect reality.

ONBOARDING.md

@@ -106,9 +106,7 @@ by an actual sytest run due to parameterised tests.
 ### Where should I put new tests?
 
 If the test *only* has CS API calls, then put it in `/tests/csapi`. If the test involves both CS API and Federation, or just Federation, put it in `/tests`.
-This is because of how parallelisation works currently. All federation tests MUST be in the same directory due to the use of shared resources (for example,
-the local Complement server always binds to `:8448` which is a problem if 2 fed tests want to do that at the same time). This will be resolved in the future
-by the use of `.well-known` but at present this is how things stand.
+This will change in the future once we have decided how to split tests by category.
 
 ### Should I always make a new blueprint for a test?
 
@@ -173,20 +171,10 @@ There is no syntactically pleasing way to do this. Create a separate function wh
 
 This is done using standard Go testing mechanisms, use `t.Logf(...)` which will be logged only if the test fails or if `-v` is set. Note that you will not need to log HTTP requests performed using one of the built in deployment clients as they are already wrapped in loggers. For full HTTP logs, use `COMPLEMENT_DEBUG=1`.
 
-For debugging, you can also use `logrus` to expand a bunch of variables:
-
-```go
-logrus.WithFields(logrus.Fields{
-	"events": events,
-	"context": context,
-}).Error("message response")
-```
-
 ### How do I show the server logs even when the tests pass?
 
 Normally, server logs are only printed when one of the tests fail. To override that behavior to always show server logs, you can use `COMPLEMENT_ALWAYS_PRINT_SERVER_LOGS=1`.
 
-
 ### How do I skip a test?
 
 To conditionally skip a *single* test based on the homeserver being run, add a single line at the start of the test:
@@ -263,6 +251,4 @@ It can be useful to view the output of a test in Element to better debug somethi
 
 ### What do I need to know if I'm coming from sytest?
 
-Sytest has a concept of a `fixture` to configure the homeserver or test in a particular way, these are replaced with a `Blueprint` in Complement.
-
-Unlike Sytest, each test must opt-in to attaching core functionality to the server so the reader can clearly see what is and is not being handled automatically.
+Unlike Sytest, each test must opt-in to attaching core functionality to the test federation server so the reader can clearly see what is and is not being handled automatically.

dockerfiles/README.md

@@ -1,8 +1,9 @@
-This directory contains a list of dockerfiles which can be used as-is with Complement. These images will pull source code directly from Github so you do not need a local checkout installed. You must run the `docker build` command inside this directory to pull in the config files into the build context.
+This directory contains a list of dockerfiles which can be used with Complement.
 
-```
-$ docker build -t complement-dendrite -f Dendrite.Dockerfile .
-```
-
-Try it out by building them and then passing them as `COMPLEMENT_BASE_IMAGE` (no args required).
+This used to have stand-alone Dockerfiles which would pull sources directly and build them for testing.
+However, this doesn't work nicely when running Complement with local checkouts, so homeservers would
+end up copying the Dockerfiles in this directory to their own repository. In an effort to reduce
+duplication, we now point to dockerfiles in respective repositories rather than have them directly here.
 
+- Dendrite: https://github.com/matrix-org/dendrite/blob/v0.6.3/build/scripts/Complement.Dockerfile
+- Synapse: https://github.com/matrix-org/synapse/blob/v1.52.0/scripts-dev/complement.sh

internal/docker/deployer.go

@@ -36,6 +36,12 @@ import (
 	"github.com/matrix-org/complement/internal/config"
 )
 
+const (
+	MountCACertPath     = "/complement/ca/ca.crt"
+	MountCAKeyPath      = "/complement/ca/ca.key"
+	MountAppServicePath = "/complement/appservice/" // All registration files sit here
+)
+
 type Deployer struct {
 	DeployNamespace string
 	Docker          *client.Client
@@ -174,18 +180,6 @@ func deployImage(
 		extraHosts = []string{HostnameRunningComplement + ":172.17.0.1"}
 	}
 
-	toMount := []Volume{
-		&VolumeAppService{},
-	}
-
-	for _, m := range toMount {
-		err = m.Prepare(ctx, docker, contextStr)
-		if err != nil {
-			return nil, fmt.Errorf("failed to prepare volume: %s", err)
-		}
-		mounts = append(mounts, m.Mount())
-	}
-
 	env := []string{
 		"SERVER_NAME=" + hsName,
 		// TODO: Remove once Synapse images don't rely on this anymore
@@ -225,7 +219,7 @@ func deployImage(
 
 	// Create the application service files
 	for asID, registration := range asIDToRegistrationMap {
-		err = copyToContainer(docker, containerID, fmt.Sprintf("/appservices/%s.yaml", url.PathEscape(asID)), []byte(registration))
+		err = copyToContainer(docker, containerID, fmt.Sprintf("%s%s.yaml", MountAppServicePath, url.PathEscape(asID)), []byte(registration))
 		if err != nil {
 			return nil, err
 		}
@@ -236,15 +230,15 @@ func deployImage(
 	if err != nil {
 		return nil, fmt.Errorf("failed to get CA certificate: %s", err)
 	}
-	err = copyToContainer(docker, containerID, "/ca/ca.crt", certBytes)
+	err = copyToContainer(docker, containerID, MountCACertPath, certBytes)
 	if err != nil {
 		return nil, fmt.Errorf("failed to copy CA certificate to container: %s", err)
 	}
 	certKeyBytes, err := cfg.CAPrivateKeyBytes()
 	if err != nil {
 		return nil, fmt.Errorf("failed to get CA key: %s", err)
 	}
-	err = copyToContainer(docker, containerID, "/ca/ca.key", certKeyBytes)
+	err = copyToContainer(docker, containerID, MountCAKeyPath, certKeyBytes)
 	if err != nil {
 		return nil, fmt.Errorf("failed to copy CA key to container: %s", err)
 	}