Next version (#540)

* Refactor asset browsing to support grouped assets

- Updated `Browse` and `passTwo` methods in `LocalAssetBrowser` to return `AssetGroup` instead of `LocalAssetFile`.
- Implemented logic to handle different types of asset groups (e.g., motion photos, single images, videos).
- Added album management for asset groups.
- Introduced `AssetGroup` struct in `group.go` to represent linked assets.
- Updated tests to validate and handle `AssetGroup` instead of individual assets.
- Removed redundant album management code from `assetFromFile` method.

* Refactor `LocalAssetFile` struct: remove metadata from asset

* Refactor metadata reading functions to return metadata and error

- Updated `ReadMetadata`, `metadataFromExiftool`, and `metadataFromDirectRead` functions to return `*metadata.Metadata` and `error` instead of setting `la.Metadata` directly.
- Adjusted logic to handle the returned metadata and error appropriately.

* Add Metadata field to AssetGroup struct in adapters/group.go

* Refactor AssetGroup struct in adapters/group.go to support grouped assets

* Refactor asset browsing to support grouped assets

* Refactor `LocalAssetFile` struct: remove metadata from asset

* Refactor Adapter interface to support browsing grouped assets

* Refactor `AssetUpload` function in immich/asset.go don't force XMP

* Refactor `UpdateAsset`  pass a structure with paramters

* Refactor `UpdateAsset` to accept a structure with parameters

* Refactor asset group handling  in upload command: manage the live photo

* Refactor file browsing in upload command to use asset groups

* Refactor date handling flags in AddDateHandlingFlags function default is DateMethodNameThenExif

* Refactor logging in Recorder.Report()

* Refactor Open()

- Add functionality to list the flags and arguments used in the command to provide more context in the logs.

* Refactor Open() in cmdFolder.go to use cmdFolder as a parameter

* Refactor Open() in cmdFolder.go to use cmdFolder as a parameter

* Refactor Open() in cmdFolder.go to use cmdFolder as a parameter

* Refactor Open() in cmdFolder.go to use cmdFolder as a parameter and bind flags to environment variables

* Refactor Open() in cmdFolder.go to use cmdFolder as a parameter and bind flags to environment variables

* Refactor Open() in cmdFolder.go to use cmdFolder as a parameter and bind flags to environment variables
Refactor NewRecorder() in fileevents.go to remove debug parameter
Refactor AddCommands() in commands.go to use context and application parameters
Refactor .goreleaser.yaml to update ldflags for version and commit
Refactor CommandTool() in tool.nogo to remove unused imports and update error message
Refactor readFolder.go to update function signature and parameter names
Refactor version.go to initialize version and commit at runtime
Refactor banner.go to update banner with version information
Refactor flagsCommon.go to remove unused flags and function
Refactor upload/TEST_DATA/Takeout3/Photos from 2023/DSC_0238.JPG.json to update creationTime and photoTakenTime
Refactor upload/TEST_DATA/Takeout3/Photos from 2023/DSC_0238_1.JPG.json to update creationTime and photoTakenTime
Refactor upload/TEST_DATA/Takeout3/Photos from 2023/DSC_0238.JPG(1).json to update creationTime and photoTakenTime
Refactor upload/TEST_DATA/Takeout3/Photos from 2023/DSC_0238.JPG to add new file
Refactor upload/TEST_DATA/Takeout3/Photos from 2023/DSC_0238_1.JPG to add new file
Refactor upload/TEST_DATA/Takeout3/Photos from 2023/DSC_0238(1).JPG to add new file
Refactor upload/TEST_DATA/Takeout2/Google Photos/Sans titre(9)/métadonnées.json to add new file
Refactor upload/TEST_DATA/Takeout1/Google Photos/Album test 6-10-23/métadonnées.json to add new file
Refactor cmdVersion/version.go to remove unused file
Refactor version/version.go to update version and commit initialization
Refactor upload/TEST_DATA/Takeout3/Photos from 2023/DSC_0238.JPG.json to update geoData
Refactor upload/TEST_DATA/Takeout3/Photos from 2023/DSC_0238_1.JPG.json to update geoData
Refactor upload/TEST_DATA/Takeout3/Photos from 2023/DSC_0238.JPG(1).json to update geoData
Refactor upload/TEST_DATA/Takeout3/Photos from 2023/DSC_0238.JPG to update title and creationTime
Refactor upload/TEST_DATA/Takeout3/Photos from 2023/DSC_0238_1.JPG to update title and creationTime
Refactor upload/TEST_DATA/Takeout3/Photos from 2023/DSC_0238(1).JPG to update title and creationTime
Refactor upload/TEST_DATA/Takeout2/Google Photos/Sans titre(9)/métadonnées.json to update title and creationTime
Refactor upload/TEST_DATA/Takeout1/Google Photos/Album test 6-10-23/métadonnées.json to update title and creationTime

* wip

* fix log in UI wip

* Refactor upload command to use separate sub-commands for different sources

- The `upload` command has been refactored to use separate sub-commands for different sources of photos.
- The `upload` command now has a `from-folder` sub-command to upload photos from a folder.
- The `upload` command also has a `from-google-photos` sub-command to upload photos from a Google Photos takeout.

* typo in upload command

* Refactor NewImmichClient() in client.go to include option for dry run

* Refactor upload command to use more descriptive wording for Google Photos source

* Refactor client.go to write dry-run indication on the log

* Refactor log.go to mask  the API key flag

* Refactor testgp_test.go to fix the order of live photos in the livePhotos map

* Refactor testgp_test.go to fix the order of live photos in the livePhotos map

* remove useless source files

* Refactor main.go to handle nil log writer when logging errors

* Refactor upload command to remove exif flags for the google-photos

* Refactor .goreleaser.yaml

* Refactor .goreleaser.yaml and go.mod files

* Refactor options.go to remove UseJSONMetadata flag
``

* Refactor upload command flag name for importing from a specific Google Photos album

* Add documentation for Immich-go environment variables

* Refactor readFolder.go to improve live photo  linking logic

* Refactor testgp_livephoto_test.go to improve live photo linking logic

* fix issue #500 (#502)

* Refactor LocalAssetFile to add LogValue method

* Refactor readFolder.go to not group photo and move with a different capture date

* Refactor readFolder.go to improve logging of associated metadata

* Refactor log.go to separate log file opening logic

* readFolder: adapt tests to write a log

* Refactor metadata.go to add LogValue method

* Refactor log.go to use assetFile struct for logging in logMessage function

* Refactor json_test.go to include dateTaken and title fields in the test cases

* Refactor json.go to simplify the  GoogleMetaData struct, and fix the logValue()

* Refactor log.go to fanout log in file and console with different handler

* Refactor fileevent package to separate FileAndName struct and add LogValue method

* Refactor googlephotos.go to improve logging and metadata handling

* Fix json.go to handle photo taken time in GoogleMetaData struct

* Refactor googlephotos.go to handle photo taken time in GoogleMetaData struct

* Refactor uploadAsset method to improve error handling and logging

* Refactor upload/ui.go to use colored logs

* Refactor log.go to use console-slog for colored logs

* Refactor ui  to display google takeout progression

* Refactor error handling in main.go

* Update dependencies in go.mod

* Discard macOS resource files
Fixes #480

* Discard macOS resource files (#504)

Fixes #480

* Refactor readFolder.go to skip banned folders during file traversal

* Merge branch 'optimization--don't-parse-banned-folders' into new-cli

* Merge branch 'new-cli' into next

* edit release.md

* Add test-config.yaml to .gitignore

.github/workflows/ci.yml

@@ -23,6 +23,9 @@ jobs:
           go-version-file: "go.mod"
           cache-dependency-path: "go.sum"
 
+      - name: Install ExifTool
+        run: sudo apt-get install -y exiftool
+
       - name: golangci-lint
         uses: golangci/golangci-lint-action@v6
         with:

.gitignore

@@ -18,3 +18,5 @@ archive.tar
 *.test
 .env
 *.env
+test-config.yaml
+__debug_*

.goreleaser.yaml

@@ -1,5 +1,7 @@
 # This is an example .goreleaser.yml file with some sensible defaults.
 # Make sure to check the documentation at https://goreleaser.com
+version: 2
+
 before:
   hooks:
     # You may remove this if you don't use go modules.
@@ -23,7 +25,7 @@ builds:
     flags:
       - -trimpath
     ldflags:
-    - '-s -w "-extldflags=-static" -X main.version={{.Version}} -X main.commit={{.Commit}} -X main.date={{.Date}} -X main.builtBy=goreleaser'
+    - '-s -w "-extldflags=-static" -X version.Version={{.Version}} -X version.Commit={{.Commit}} -X Version.Date={{.Date}} -X main.builtBy=goreleaser'
 
 archives:
   - format: tar.gz
@@ -54,3 +56,191 @@ changelog:
 # Feel free to remove those if you don't want/use them.
 # yaml-language-server: $schema=https://goreleaser.com/static/schema.json
 # vim: set ts=2 sw=2 tw=0 fo=cnqoj
+
+# .goreleaser.yaml
+release:
+  # Repo in which the release will be created.
+  # Default: extracted from the origin remote URL or empty if its private hosted.
+  # github:
+  #   owner: user
+  #   name: repo
+
+  # IDs of the archives to use.
+  # Empty means all IDs.
+  #
+  # Default: [].
+  # ids:
+  #   - foo
+  #   - bar
+
+  # If set to true, will not auto-publish the release.
+  # Note: all GitHub releases start as drafts while artifacts are uploaded.
+  # Available only for GitHub and Gitea.
+  draft: true
+
+  # Whether to remove existing draft releases with the same name before creating
+  # a new one.
+  #
+  # Only effective if `draft` is set to true.
+  # Available only for GitHub.
+  replace_existing_draft: true
+
+  # Whether to remove an artifact that already exists.
+  #
+  # Available only for GitHub.
+  # This might be a bit expensive (rate-limiting speaking), so it is only done
+  # when the upload of an artifact fails with a 422 (which means it already
+  # exists in the release).
+  # We then grab the list of artifacts from the release, and delete the file
+  # that matches the one we're trying to upload.
+  # GoReleaser will then retry its upload.
+  replace_existing_artifacts: true
+
+  # Useful if you want to delay the creation of the tag in the remote.
+  # You can create the tag locally, but not push it, and run GoReleaser.
+  # It'll then set the `target_commitish` portion of the GitHub release to the
+  # value of this field.
+  # Only works on GitHub.
+  #
+  # Default: ''.
+  # Templates: allowed.
+  target_commitish: "{{ .Commit }}"
+
+  # This allows to change which tag GitHub will create.
+  # Usually you'll use this together with `target_commitish`, or if you want to
+  # publish a binary from a monorepo into a public repository somewhere, without
+  # the tag prefix.
+  #
+  # This feature is only available in GoReleaser Pro.
+  # Default: '{{ .PrefixedCurrentTag }}'.
+  # Templates: allowed.
+  # tag: "{{ .CurrentTag }}"
+
+  # If set, will create a release discussion in the category specified.
+  #
+  # Warning: do not use categories in the 'Announcement' format.
+  #  Check https://github.com/goreleaser/goreleaser/issues/2304 for more info.
+  #
+  # Default: ''.
+  # discussion_category_name: General
+
+  # If set to auto, will mark the release as not ready for production
+  # in case there is an indicator for this in the tag e.g. v1.0.0-rc1
+  # If set to true, will mark the release as not ready for production.
+  # Default: false.
+  prerelease: auto
+
+  # If set to false, will NOT mark the release as "latest".
+  # This prevents it from being shown at the top of the release list,
+  # and from being returned when calling https://api.github.com/repos/OWNER/REPO/releases/latest.
+  #
+  # Available only for GitHub.
+  #
+  # Default: true.
+  make_latest: true
+
+  # What to do with the release notes in case there the release already exists.
+  #
+  # Valid options are:
+  # - `keep-existing`: keep the existing notes
+  # - `append`: append the current release notes to the existing notes
+  # - `prepend`: prepend the current release notes to the existing notes
+  # - `replace`: replace existing notes
+  #
+  # Default: `keep-existing`.
+  mode: append
+
+  # Header for the release body.
+  #
+  # Templates: allowed.
+  # header: |
+  #   ## Some title ({{ .Date }})
+
+  #   Welcome to this new release!
+
+  # Header for the release body.
+  #
+  # This feature is only available in GoReleaser Pro.
+  # header:
+  #   # Loads from an URL.
+  #   from_url:
+  #     # Templates: allowed.
+  #     url: https://foo.bar/header.md
+  #     headers:
+  #       x-api-token: "${MYCOMPANY_TOKEN}"
+
+  #   # Loads from a local file.
+  #   # Overrides `from_url`.
+  #   from_file:
+  #     # Templates: allowed.
+  #     path: ./header.md
+
+  # Footer for the release body.
+  #
+  # Templates: allowed.
+  # footer: |
+  #   ## Thanks
+
+  #   Those were the changes on {{ .Tag }}!
+
+  # Footer for the release body.
+  #
+  # This feature is only available in GoReleaser Pro.
+  # footer:
+  #   # Loads from an URL.
+  #   from_url:
+  #     # Templates: allowed.
+  #     url: https://foo.bar/footer.md
+  #     footers:
+  #       x-api-token: "${MYCOMPANY_TOKEN}"
+
+  #   # Loads from a local file.
+  #   # Overrides `from_url`.
+  #   from_file:
+  #     # Templates: allowed.
+  #     path: ./footer.md
+
+  # # You can change the name of the release.
+  # #
+  # # Default: '{{.Tag}}' ('{{.PrefixedTag}}' on Pro).
+  # # Templates: allowed.
+  # name_template: "{{.ProjectName}}-v{{.Version}} {{.Env.USER}}"
+
+  # You can disable this pipe in order to not create the release on any SCM.
+  # Keep in mind that this might also break things that depend on the release
+  # URL, for instance, homebrew taps.
+  #
+  # Templates: allowed.
+  disable: true
+
+  # Set this to true if you want to disable just the artifact upload to the SCM.
+  # If this is true, GoReleaser will still create the release with the
+  # changelog, but won't upload anything to it.
+  #
+  # Templates: allowed.
+  skip_upload: true
+
+  # You can add extra pre-existing files to the release.
+  # The filename on the release will be the last part of the path (base).
+  # If another file with the same name exists, the last one found will be used.
+  #
+  # Templates: allowed.
+  # extra_files:
+  #   - glob: ./path/to/file.txt
+  #   - glob: ./glob/**/to/**/file/**/*
+  #   - glob: ./glob/foo/to/bar/file/foobar/override_from_previous
+  #   - glob: ./single_file.txt
+  #     name_template: file.txt # note that this only works if glob matches 1 file only
+
+  # Additional templated extra files to add to the release.
+  # Those files will have their contents pass through the template engine,
+  # and its results will be added to the release.
+  #
+  # This feature is only available in GoReleaser Pro.
+  # Templates: allowed.
+  # templated_extra_files:
+  #   - src: LICENSE.tpl
+  #     dst: LICENSE.txt
+
+  # # Upload metadata.json and artifacts.json to the release as well.
+  # include_meta: true

adapters/adpaters.go

@@ -0,0 +1,16 @@
+package adapters
+
+import (
+	"context"
+
+	"github.com/simulot/immich-go/internal/assets"
+)
+
+type Reader interface {
+	Browse(cxt context.Context) chan *assets.Group
+}
+
+type AssetWriter interface {
+	WriteAsset(context.Context, *assets.Asset) error
+	// WriteGroup(ctx context.Context, group *assets.Group) error
+}

adapters/folder/options.go

@@ -0,0 +1,151 @@
+package folder
+
+import (
+	"fmt"
+	"strings"
+	"time"
+
+	cliflags "github.com/simulot/immich-go/internal/cliFlags"
+	"github.com/simulot/immich-go/internal/filenames"
+	"github.com/simulot/immich-go/internal/filetypes"
+	"github.com/simulot/immich-go/internal/filters"
+	"github.com/simulot/immich-go/internal/namematcher"
+	"github.com/spf13/cobra"
+)
+
+// ImportFolderOptions represents the flags used for importing assets from a file system.
+type ImportFolderOptions struct {
+	// UsePathAsAlbumName determines whether to create albums based on the full path to the asset.
+	UsePathAsAlbumName AlbumFolderMode
+
+	// AlbumNamePathSeparator specifies how multiple (sub) folders are joined when creating album names.
+	AlbumNamePathSeparator string
+
+	// ImportIntoAlbum is the name of the album where all assets will be added.
+	ImportIntoAlbum string
+
+	// BannedFiles is a list of file name patterns to be excluded from the import process.
+	BannedFiles namematcher.List
+
+	// Recursive indicates whether to explore the folder and all its sub-folders.
+	Recursive bool
+
+	// InclusionFlags controls the file extensions to be included in the import process.
+	InclusionFlags cliflags.InclusionFlags
+
+	// // ExifToolFlags specifies options for the exif.
+	// ExifToolFlags exif.ExifToolFlags
+
+	// IgnoreSideCarFiles indicates whether to ignore XMP files during the import process.
+	IgnoreSideCarFiles bool
+
+	// Stack jpg/raw
+	StackJpgWithRaw bool
+
+	// Stack burst
+	StackBurstPhotos bool
+
+	// SupportedMedia is the server's actual list of supported media types.
+	SupportedMedia filetypes.SupportedMedia
+
+	// InfoCollector is used to extract information from the file name.
+	InfoCollector *filenames.InfoCollector
+
+	// ManageHEICJPG determines whether to manage HEIC to JPG conversion options.
+	ManageHEICJPG filters.HeicJpgFlag
+
+	// ManageRawJPG determines how to manage raw and JPEG files.
+	ManageRawJPG filters.RawJPGFlag
+
+	// BurstFlag determines how to manage burst photos.
+	ManageBurst filters.BurstFlag
+
+	// ManageEpsonFastFoto enables the management of Epson FastFoto files.
+	ManageEpsonFastFoto bool
+
+	// Tags is a list of tags to be added to the imported assets.
+	Tags []string
+
+	// Folder as tags
+	FolderAsTags bool
+
+	// SessionTag indicates whether to add a session tag to the imported assets.
+	SessionTag bool
+	session    string // Session tag value
+
+	// TakeDateFromFilename indicates whether to take the date from the filename if the date isn't available in the image.
+	TakeDateFromFilename bool
+
+	// local time zone
+	TZ *time.Location
+}
+
+func (o *ImportFolderOptions) AddFromFolderFlags(cmd *cobra.Command, parent *cobra.Command) {
+	o.ManageHEICJPG = filters.HeicJpgNothing
+	o.ManageRawJPG = filters.RawJPGNothing
+	o.ManageBurst = filters.BurstNothing
+	o.Recursive = true
+	o.SupportedMedia = filetypes.DefaultSupportedMedia
+	o.UsePathAsAlbumName = FolderModeNone
+	o.BannedFiles, _ = namematcher.New(
+		`@eaDir/`,
+		`@__thumb/`,          // QNAP
+		`SYNOFILE_THUMB_*.*`, // SYNOLOGY
+		`Lightroom Catalog/`, // LR
+		`thumbnails/`,        // Android photo
+		`.DS_Store/`,         // Mac OS custom attributes
+		`._*.*`,              // MacOS resource files
+	)
+	cmd.Flags().StringVar(&o.ImportIntoAlbum, "into-album", "", "Specify an album to import all files into")
+	cmd.Flags().Var(&o.UsePathAsAlbumName, "folder-as-album", "Import all files in albums defined by the folder structure. Can be set to 'FOLDER' to use the folder name as the album name, or 'PATH' to use the full path as the album name")
+	cmd.Flags().StringVar(&o.AlbumNamePathSeparator, "album-path-joiner", " / ", "Specify a string to use when joining multiple folder names to create an album name (e.g. ' ',' - ')")
+	cmd.Flags().BoolVar(&o.Recursive, "recursive", true, "Explore the folder and all its sub-folders")
+	cmd.Flags().Var(&o.BannedFiles, "ban-file", "Exclude a file based on a pattern (case-insensitive). Can be specified multiple times.")
+	cmd.Flags().BoolVar(&o.IgnoreSideCarFiles, "ignore-sidecar-files", false, "Don't upload sidecar with the photo.")
+
+	cmd.Flags().StringSliceVar(&o.Tags, "tag", nil, "Add tags to the imported assets. Can be specified multiple times. Hierarchy is supported using a / separator (e.g. 'tag1/subtag1')")
+	cmd.Flags().BoolVar(&o.FolderAsTags, "folder-as-tags", false, "Use the folder structure as tags, (ex: the file  holiday/summer 2024/file.jpg will have the tag holiday/summer 2024)")
+	cmd.Flags().BoolVar(&o.SessionTag, "session-tag", false, "Tag uploaded photos with a tag \"{immich-go}/YYYY-MM-DD HH-MM-SS\"")
+
+	cliflags.AddInclusionFlags(cmd, &o.InclusionFlags)
+	cmd.Flags().BoolVar(&o.TakeDateFromFilename, "date-from-name", true, "Use the date from the filename if the date isn't available in the metadata (Only for .jpg,mp4,.heic,.dng,cr2,.cr3,).")
+
+	// exif.AddExifToolFlags(cmd, &o.ExifToolFlags) // disabled for now
+
+	if parent != nil && parent.Name() == "upload" {
+		cmd.Flags().Var(&o.ManageHEICJPG, "manage-heic-jpeg", "Manage coupled HEIC and JPEG files. Possible values: KeepHeic, KeepJPG, StackCoverHeic, StackCoverJPG")
+		cmd.Flags().Var(&o.ManageRawJPG, "manage-raw-jpeg", "Manage coupled RAW and JPEG files. Possible values: KeepRaw, KeepJPG, StackCoverRaw, StackCoverJPG")
+		cmd.Flags().Var(&o.ManageBurst, "manage-burst", "Manage burst photos. Possible values: Stack, StackKeepRaw, StackKeepJPEG")
+		cmd.Flags().BoolVar(&o.ManageEpsonFastFoto, "manage-epson-fastfoto", false, "Manage Epson FastFoto file (default: false)")
+	}
+}
+
+// AlbumFolderMode represents the mode in which album folders are organized.
+// Implement the interface pflag.Value
+
+type AlbumFolderMode string
+
+const (
+	FolderModeNone   AlbumFolderMode = "NONE"
+	FolderModeFolder AlbumFolderMode = "FOLDER"
+	FolderModePath   AlbumFolderMode = "PATH"
+)
+
+func (m AlbumFolderMode) String() string {
+	return string(m)
+}
+
+func (m *AlbumFolderMode) Set(v string) error {
+	v = strings.TrimSpace(strings.ToUpper(v))
+	switch v {
+	case string(FolderModeFolder), string(FolderModePath):
+		*m = AlbumFolderMode(v)
+	default:
+		return fmt.Errorf("invalid value for folder mode, expected %s, %s or %s", FolderModeFolder, FolderModePath, FolderModeNone)
+	}
+	return nil
+}
+
+func (m AlbumFolderMode) Type() string {
+	return "folderMode"
+}