-
Notifications
You must be signed in to change notification settings - Fork 727
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Split serialisation from IO #5049
Merged
Merged
Changes from all commits
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
3b967ee
Move OutputFile to cardano-api
newhoggy c9fe62b
New functions for writing to file and stdout:
newhoggy 2e5adc2
Split callsite of writeFileTextEnvelope to separate serialisation fro…
newhoggy 15a843d
Move writeFileWithOwnerPermissions to Cardano.Api.IO.Compat so that C…
newhoggy 7b91bf5
Rename writeFileWithOwnerPermissions to writeLazyByteStringFileWithOw…
newhoggy 971156d
Generalise writeLazyByteStringFileWithOwnerPermissions to handleFileF…
newhoggy 9076213
New functions for completeness:
newhoggy 2d11855
Remove writeFileTextEnvelopeWithOwnerPermissions. Use writeLazyByteS…
newhoggy File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,161 @@ | ||
| {-# LANGUAGE CPP #-} | ||
| {-# LANGUAGE DeriveGeneric #-} | ||
| {-# LANGUAGE DerivingStrategies #-} | ||
| {-# LANGUAGE GeneralizedNewtypeDeriving #-} | ||
| {-# LANGUAGE ScopedTypeVariables #-} | ||
|
|
||
| module Cardano.Api.IO | ||
| ( OutputFile(..) | ||
|
|
||
| , writeByteStringFileWithOwnerPermissions | ||
| , writeByteStringFile | ||
| , writeByteStringOutput | ||
|
|
||
| , writeLazyByteStringFileWithOwnerPermissions | ||
| , writeLazyByteStringFile | ||
| , writeLazyByteStringOutput | ||
|
|
||
| , writeTextFileWithOwnerPermissions | ||
| , writeTextFile | ||
| , writeTextOutput | ||
|
|
||
| ) where | ||
|
|
||
| #if !defined(mingw32_HOST_OS) | ||
| #define UNIX | ||
| #endif | ||
|
|
||
| #ifdef UNIX | ||
| import Control.Exception (IOException, bracket, bracketOnError, try) | ||
| import System.Directory () | ||
| import System.IO (hClose) | ||
| import System.Posix.Files (ownerModes, setFdOwnerAndGroup) | ||
| import System.Posix.IO (OpenMode (..), closeFd, defaultFileFlags, fdToHandle, openFd) | ||
| import System.Posix.User (getRealUserID) | ||
| #else | ||
| import Control.Exception (bracketOnError) | ||
| import System.Directory (removeFile, renameFile) | ||
| import System.FilePath (splitFileName, (<.>)) | ||
| import System.IO (hClose, openTempFile) | ||
| #endif | ||
|
|
||
| import Cardano.Api.Error (FileError (..)) | ||
|
|
||
| import Control.Monad.Except (runExceptT) | ||
| import Control.Monad.IO.Class (MonadIO (..)) | ||
| import Control.Monad.Trans.Except.Extra (handleIOExceptT) | ||
| import Data.Aeson.Types (FromJSON, ToJSON) | ||
| import Data.ByteString (ByteString) | ||
| import qualified Data.ByteString.Char8 as BS | ||
| import qualified Data.ByteString.Char8 as BSC | ||
| import qualified Data.ByteString.Lazy as LBS | ||
| import qualified Data.ByteString.Lazy as LBSC | ||
| import Data.String (IsString) | ||
| import Data.Text (Text) | ||
| import qualified Data.Text.IO as Text | ||
| import GHC.Generics (Generic) | ||
| import System.IO (Handle) | ||
|
|
||
| handleFileForWritingWithOwnerPermission | ||
| :: FilePath | ||
| -> (Handle -> IO ()) | ||
| -> IO (Either (FileError ()) ()) | ||
| handleFileForWritingWithOwnerPermission path f = do | ||
| #ifdef UNIX | ||
| -- On a unix based system, we grab a file descriptor and set ourselves as owner. | ||
| -- Since we're holding the file descriptor at this point, we can be sure that | ||
| -- what we're about to write to is owned by us if an error didn't occur. | ||
| user <- getRealUserID | ||
| ownedFile <- try $ | ||
| -- We only close the FD on error here, otherwise we let it leak out, since | ||
| -- it will be immediately turned into a Handle (which will be closed when | ||
| -- the Handle is closed) | ||
| bracketOnError | ||
| (openFd path WriteOnly (Just ownerModes) defaultFileFlags) | ||
| closeFd | ||
| (\fd -> setFdOwnerAndGroup fd user (-1) >> pure fd) | ||
| case ownedFile of | ||
| Left (err :: IOException) -> do | ||
| pure $ Left $ FileIOError path err | ||
| Right fd -> do | ||
| bracket | ||
| (fdToHandle fd) | ||
| hClose | ||
| (runExceptT . handleIOExceptT (FileIOError path) . f) | ||
| #else | ||
| -- On something other than unix, we make a _new_ file, and since we created it, | ||
| -- we must own it. We then place it at the target location. Unfortunately this | ||
| -- won't work correctly with pseudo-files. | ||
| bracketOnError | ||
| (openTempFile targetDir $ targetFile <.> "tmp") | ||
| (\(tmpPath, h) -> do | ||
| hClose h >> removeFile tmpPath | ||
| return . Left $ FileErrorTempFile path tmpPath h) | ||
| (\(tmpPath, h) -> do | ||
| f h | ||
| hClose h | ||
| renameFile tmpPath path | ||
| return $ Right ()) | ||
| where | ||
| (targetDir, targetFile) = splitFileName path | ||
| #endif | ||
|
|
||
| newtype OutputFile = OutputFile | ||
| { unOutputFile :: FilePath | ||
| } | ||
| deriving Generic | ||
| deriving newtype (Eq, Ord, Show, IsString, ToJSON, FromJSON) | ||
|
|
||
| writeByteStringFile :: MonadIO m => FilePath -> ByteString -> m (Either (FileError ()) ()) | ||
| writeByteStringFile fp bs = runExceptT $ | ||
| handleIOExceptT (FileIOError fp) $ BS.writeFile fp bs | ||
|
|
||
| writeByteStringFileWithOwnerPermissions | ||
| :: FilePath | ||
| -> BS.ByteString | ||
| -> IO (Either (FileError ()) ()) | ||
| writeByteStringFileWithOwnerPermissions fp bs = | ||
| handleFileForWritingWithOwnerPermission fp $ \h -> | ||
| BS.hPut h bs | ||
|
|
||
| writeByteStringOutput :: MonadIO m => Maybe FilePath -> ByteString -> m (Either (FileError ()) ()) | ||
| writeByteStringOutput mOutput bs = runExceptT $ | ||
| case mOutput of | ||
| Just fp -> handleIOExceptT (FileIOError fp) $ BS.writeFile fp bs | ||
| Nothing -> liftIO $ BSC.putStr bs | ||
|
|
||
| writeLazyByteStringFile :: MonadIO m => FilePath -> LBS.ByteString -> m (Either (FileError ()) ()) | ||
| writeLazyByteStringFile fp bs = runExceptT $ | ||
| handleIOExceptT (FileIOError fp) $ LBS.writeFile fp bs | ||
|
|
||
| writeLazyByteStringFileWithOwnerPermissions | ||
| :: FilePath | ||
| -> LBS.ByteString | ||
| -> IO (Either (FileError ()) ()) | ||
| writeLazyByteStringFileWithOwnerPermissions fp lbs = | ||
| handleFileForWritingWithOwnerPermission fp $ \h -> | ||
| LBS.hPut h lbs | ||
|
|
||
| writeLazyByteStringOutput :: MonadIO m => Maybe FilePath -> LBS.ByteString -> m (Either (FileError ()) ()) | ||
| writeLazyByteStringOutput mOutput bs = runExceptT $ | ||
| case mOutput of | ||
| Just fp -> handleIOExceptT (FileIOError fp) $ LBS.writeFile fp bs | ||
| Nothing -> liftIO $ LBSC.putStr bs | ||
|
|
||
| writeTextFile :: MonadIO m => FilePath -> Text -> m (Either (FileError ()) ()) | ||
| writeTextFile fp t = runExceptT $ | ||
| handleIOExceptT (FileIOError fp) $ Text.writeFile fp t | ||
|
|
||
| writeTextFileWithOwnerPermissions | ||
| :: FilePath | ||
| -> Text | ||
| -> IO (Either (FileError ()) ()) | ||
| writeTextFileWithOwnerPermissions fp t = | ||
| handleFileForWritingWithOwnerPermission fp $ \h -> | ||
| Text.hPutStr h t | ||
|
|
||
| writeTextOutput :: MonadIO m => Maybe FilePath -> Text -> m (Either (FileError ()) ()) | ||
| writeTextOutput mOutput t = runExceptT $ | ||
| case mOutput of | ||
| Just fp -> handleIOExceptT (FileIOError fp) $ Text.writeFile fp t | ||
| Nothing -> liftIO $ Text.putStr t | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The first argument is a bit unclear, it takes a moment to understand that
Nothingmeansstdout. Maybe encoding that information in the type would be better? For example:where
Handlecan bestdoutorstderr(which doesn't really matter here).Consequently, if you open a handle to a file path, I think the other functions, which then could operate on
Handles, could be unified and simplified I think (instead of matchingMaybein every one of them).There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There have been multiple attempts to do such a thing, but they've been piecemeal.
The purpose of this PR is not to introduce such an abstraction. We already have code that uses
Maybe FilePathand keeping this code to that convention minimises difference.We may introduce an appropriate abstraction in a separate PR and do it across the entire codebase to handle consistently.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@carbolymer this would be a good issue to create and tackle next.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've created: #5062