From 7b1033b85f2c79e9f13759ae71fb3874ee0b9026 Mon Sep 17 00:00:00 2001 From: Erick Oduniyi Date: Wed, 19 Aug 2020 03:06:01 -0500 Subject: [PATCH 01/23] Change "I" to "We" to make it sound like a team project --- hyperSpec/vignettes/fileio.Rmd | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/hyperSpec/vignettes/fileio.Rmd b/hyperSpec/vignettes/fileio.Rmd index 2ae9a33b4..1cb189837 100644 --- a/hyperSpec/vignettes/fileio.Rmd +++ b/hyperSpec/vignettes/fileio.Rmd @@ -210,7 +210,7 @@ Overview lists of the directly supported file formats are in the appendix: sorte 1. `# NOTE:`{.r} Do not remove `\index{}` tags. They might be useful in the future. -2. I suggest cheating a specialized function, e.g., `hyperSpec()`{.r} that wrapps functionality of `new("hyperSpec", spc, wavelength, data, labels)`{.r}. +2. We suggest cheating a specialized function, e.g., `hyperSpec()`{.r} that wrapps functionality of `new("hyperSpec", spc, wavelength, data, labels)`{.r}. ``` @@ -244,7 +244,7 @@ With the arguments: Thus, once your data is in R's workspace, creating a `hyperSpec`{.r} object is easy. -I suggest wrapping the code to import your data and the line joining it into a `hyperSpec`{.r} object by your own import function. +We suggest wrapping the code to import your data and the line joining it into a `hyperSpec`{.r} object by your own import function. You are more than welcome to contribute such import code to package **hyperSpec**. Secion \@ref(sec:writing-Import) discusses examples of custom import functions. @@ -781,7 +781,7 @@ However, if you just post-process the `hyperSpec`{.r} object returned by `read.s Many spectrometer manufacturers provide a function to export their spectra into ASCII files. The functions discussed above are written in a very general way, and are highly customizable. -I recommend wrapping these calls with the appropriate settings for your spectra format in an import function. +We recommend wrapping these calls with the appropriate settings for your spectra format in an import function. Please consider contributing such import filters to package **hyperSpec**: send me the documented code (for details see the box at the beginning of this document). If you are able to import data of any format not mentioned in this document (even without the need of new converters), please let me know (details again in the box at the beginning of this document). From 36a85990629e80f1f5fc5da3ad4bc6213d78201f Mon Sep 17 00:00:00 2001 From: Erick Oduniyi Date: Wed, 19 Aug 2020 03:16:01 -0500 Subject: [PATCH 02/23] Update fileio.Rmd Check for wordiness and weird English rules that linguist care about. --- hyperSpec/vignettes/fileio.Rmd | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/hyperSpec/vignettes/fileio.Rmd b/hyperSpec/vignettes/fileio.Rmd index 1cb189837..1b3b97791 100644 --- a/hyperSpec/vignettes/fileio.Rmd +++ b/hyperSpec/vignettes/fileio.Rmd @@ -130,8 +130,8 @@ And the current names could be deprecated. From now on, all import functions have names starting with `read.\*()`. All functions previously named `scan.\*()` have been renamed accordingly. -Package **hyperSpec** supports a number of file formats relevant for different types of spectroscopy. -This is naturally only a subset of the file formats produced by different spectroscopic equipment. +Package **hyperSpec** supports several file formats relevant for different types of spectroscopy. +This file format is naturally only a subset of the file formats produced by different spectroscopic equipment. If you use package **hyperSpec** with data formats not mentioned in this document, please open a [new issue](https://github.com/cbeleites/hyperSpec/issues){target="_blank"} in **hyperSpec**'s GitHub repository so that this document can be updated. The information should include: @@ -140,7 +140,7 @@ The information should include: - Spectrometer model, manufacturer, and software, - The "native" file format (including a sample file), - Description of relevant procedures to convert the file, -- R code to import the data together with an example file that can actually be read by R, +- R code to import the data together with an example file that is read by R, - Documentation, particularly the description of the data format. From e560480554aecf8e8cb08f9fd698e216dbbad1c4 Mon Sep 17 00:00:00 2001 From: Erick Oduniyi Date: Wed, 19 Aug 2020 03:21:13 -0500 Subject: [PATCH 03/23] Fixed wordiness, mispellings, and grammar This doesn't make a difference to me, but should we avoid personal pronouns ("you", "I", etc.) --- hyperSpec/vignettes/fileio.Rmd | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/hyperSpec/vignettes/fileio.Rmd b/hyperSpec/vignettes/fileio.Rmd index 1b3b97791..999568e13 100644 --- a/hyperSpec/vignettes/fileio.Rmd +++ b/hyperSpec/vignettes/fileio.Rmd @@ -155,7 +155,7 @@ If you need help finding out how to import your data, please search and eventual 1. `# FIXME:`{.r} After the translation is completed, the contents in the box below must be fixed. 2. `# FIXME:`{.r} do not mention `vignettes.def` -3. Check if links point to correct destitations. Links to external websites must start with `https://` or `http://`. +3. Check if links point to correct destinations. Links to external websites must start with `https://` or `http://`. ``` @@ -164,7 +164,7 @@ If you need help finding out how to import your data, please search and eventual ```{block, type="note", echo=TRUE} **Reproducing the Examples in this Manual** -To run the code examples, you should create a folder named "fileio" in your working directory and copy the contents of directory . +To run the code examples, create a folder named "fileio" in your working directory and copy the contents of the directory . This online directory contains the required datasets (via `git-lfs`). ``` From 4d66ca63e4443e5985d811d9ef25e9e9c6b05353 Mon Sep 17 00:00:00 2001 From: Erick Oduniyi Date: Wed, 19 Aug 2020 03:31:09 -0500 Subject: [PATCH 04/23] Fixed mispellings and grammar --- hyperSpec/vignettes/fileio.Rmd | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/hyperSpec/vignettes/fileio.Rmd b/hyperSpec/vignettes/fileio.Rmd index 999568e13..79a01b2d7 100644 --- a/hyperSpec/vignettes/fileio.Rmd +++ b/hyperSpec/vignettes/fileio.Rmd @@ -177,18 +177,18 @@ library(hyperSpec) # Introduction -This document describes how spectra can be imported into `hyperSpec`{.r} objects. -Some possibilities to export `hyperSpec`{.r} objects as files are mentioned, too. +This document describes how to import spectra into `hyperSpec`{.r} objects. +We discuss exporting `hyperSpec`{.r} objects as files as well. The most basic funtion to create `hyperSpec`{.r} objects is `new("hyperSpec")`{.r} (section \@ref(sec:new)). It makes a `hyperSpec`{.r} object from data already in R's workspace. -Thus, once the spectra are imported into R, conversion to `hyperSpec`{.r} objects is straightforward. +Thus, after spectra importation into R, conversion to `hyperSpec`{.r} objects is straightforward. -In addition, package **hyperSpec** comes with predifined import functions for different data formats. +Additionally, package **hyperSpec** comes with predefined import functions for different data formats. This document divides the discussion into dealing with ASCII files (section \@ref(sec:ascii)) and binary file formats (section \@ref(sec:binary-file-formats)). -If data export for the respective format is possible, it is discussed in the same sections. +If data export for the respective format is possible, we discuss it in the same sections. As sometimes the actual data written by the spectrometer software exhibits peculiarities, package **hyperSpec** offers several specialized import functions. -These are in general named after the data format followed by the manufacturer (e.g., `read.ENVI.Nicolet`). +In general, the naming convention is the data format followed by the manufacturer (e.g., `read.ENVI.Nicolet`). ```{block, type="note-t", echo=show_reviewers_notes} From bfc474c14c1aa79f0661d3a107bec2f94eabf9f2 Mon Sep 17 00:00:00 2001 From: Erick Oduniyi Date: Wed, 19 Aug 2020 11:47:20 -0500 Subject: [PATCH 05/23] Edit "#Writing your own..." and appendix section --- hyperSpec/vignettes/fileio.Rmd | 41 +++++++++++++++++----------------- 1 file changed, 21 insertions(+), 20 deletions(-) diff --git a/hyperSpec/vignettes/fileio.Rmd b/hyperSpec/vignettes/fileio.Rmd index 79a01b2d7..8a25a8953 100644 --- a/hyperSpec/vignettes/fileio.Rmd +++ b/hyperSpec/vignettes/fileio.Rmd @@ -243,9 +243,9 @@ With the arguments: ---------------- --------------------------------------------------------------------------- -Thus, once your data is in R's workspace, creating a `hyperSpec`{.r} object is easy. -We suggest wrapping the code to import your data and the line joining it into a `hyperSpec`{.r} object by your own import function. -You are more than welcome to contribute such import code to package **hyperSpec**. +hus, once the data is in R's workspace, creating a `hyperSpec`{.r} object is easy. +We suggest wrapping the code to import the data and the line joining it into a `hyperSpec`{.r} object by a user-created import function. +Users are more than welcome to contribute such import code to package **hyperSpec**. Secion \@ref(sec:writing-Import) discusses examples of custom import functions. @@ -1199,10 +1199,10 @@ read.txt.Witec.Graph("fileio/txt.Witec/nofilename (Header).txt", encoding = "lat This function reads the spectra files automatically, if they are named properly and extracts additional information of the header file. As for the other Witec functions it is possible to read image data by by selecting `type = "map"`{.r}. Line scans and z-stacks should be read as single spectra. -# Writing your own Import Function {#sec:writing-Import} + unction {#sec:writing-Import} -This section gives examples how to write import functions. The first example implements an import -filter for an ASCII file format basically from scratch. The second example shows how to implement +This section gives examples of how to write import functions. The first example implements an import +filter for an ASCII file format, basically from scratch. The second example shows how to implement more details for an already existing import filter. @@ -1215,10 +1215,10 @@ more details for an already existing import filter. \index{PerkinElmer!Fluorescence} \index{PerkinElmer!ASCII long} -The raw spectra of the `flu`{.r} data set (see also the respective vignette) are in PerkinElmer's ASCII file format, one spectrum per file. +The raw spectra of the `flu`{.r} data set (see also the separate vignette) are in PerkinElmer's ASCII file format, one spectrum per file. We need a function that automatically reads all files specified by a pattern, such as `*.txt`{.r}. -In order to gain speed, the spectra matrix should be preallocated after the first file is read. +To gain speed, **hyperSpec** preallocates the spectra matrix after the first reading of the file. @@ -1232,22 +1232,22 @@ In order to gain speed, the spectra matrix should be preallocated after the firs A short examination of the files (`flu*.txt`{.r} in directory `txt.PerkinElmer`) reveals that the actual spectrum starts at line 55, after a line containing `\#DATA`. -For now, no other information of the files is to be extracted. +For now, no other information about the files is to be extracted. It is thus easier to skip the first 54 lines than searching for the line after `\#DATA`. -A fully featured import function should support: +A fully-featured import function should support: - Reading multiple files by giving a pattern - hand further arguments to `scan()`{.r}. This comes handy in case the function is used later to import other data types. -- Also skipping 54 lines would be a weird default, so we rather require it to be given +- Also, skipping 54 lines would be a weird default, so we instead require it to be given explicitly. -- The same applies for the axis labels: they should default to reasonable settings for -fluorescence spectra, but it should be possible to change them if needed. +- The same applies to the axis labels: they should default to reasonable settings for +fluorescence spectra, but it should be possible to change them. - The usual log entry arguments should be supplied. - A sanity check should be implemented: stop with an error if a file does not have the same wavelength axis as the others. -- If no file can be found, an empty `hyperSpec`{.r} object is a reasonable result: There is no need +- If no file can be found, an empty `hyperSpec`{.r} object is a consistent result: There is no need to stop with an error, but it is polite to issue an additional warning. - Finally, package **hyperSpec** does some optional common post-processing for all imported files such as attaching the filename (or connection description) to the `hyperSpec`{.r} object (column `filename`{.r}) and deleting empty spectra. These options can be globally switched on or off by options. @@ -1259,8 +1259,9 @@ to stop with an error, but it is polite to issue an additional warning. writeLines(readLines("read.txt.PerkinElmer.R")) ``` +imports the spectra. Note how the labels are set. -The label with the special name `.wavelength`{.r} corresponds to the wavelength axis, all data columns should have a label with the same name. +The label with the unique name `.wavelength`{.r} corresponds to the wavelength axis, all data columns should have a label with the same name. The spectra are always in a data column called `spc`{.r}. Thus, @@ -1272,7 +1273,7 @@ read.txt.PerkinElmer(Sys.glob("fileio/txt.PerkinElmer/flu?.txt"), skip = 54) imports the spectra. -This function is not exported by package **hyperSpec**: while it is already useful for importing files, it is not yet general enough to work immediately with new data, e.g., the the file header is completely ignored. +The **hyperSpec** package does not export this function: while it is already useful for importing files, it is not general enough to work immediately with new data, e.g., completely ignoring the file header. Thus information like the excitation wavelength is lost. @@ -1288,14 +1289,14 @@ Thus information like the excitation wavelength is lost. \index{Nicolet!Map} \index{Nicolet!Infrared} -The function `read.ENVI.Nicolet()`{.r} is a good example for a more specific import filter derived from a general filter for the respective file type. +The function `read.ENVI.Nicolet()`{.r} is an excellent example of a more specific import filter derived from a generic filter for the particular file type. Nicolet FT-IR Imaging software saves some non-standard keywords in the header file of the ENVI data. -These information can be used to reconstruct the $x$ and $y$ axes of the images. +This information can be used to reconstruct the $x$ and $y$ axes of the images. The units of the spectra are saved as well. Function `read.ENVI.Nicolet()`{.r} thus first adjusts the parameters for `read.ENVI()`{.r}. Then `read.ENVI()`{.r} does the main work of importing the file. -The resulting `hyperSpec`{.r} object is post-processed according to the special header entries. +The resulting `hyperSpec`{.r} object is post-processed according to the unique header entries. For using the function, see section \@ref(sec:read-ENVI-Nicolet). @@ -1308,7 +1309,7 @@ writeLines(readLines("read.ENVI.Nicolet.R")) ## Deriving import filters for `spc` files -Please note that future changes inside the `read.spc()`{.r} function are likely to occur. +Please note that future changes inside the `read.spc()`{.r} function is likely to occur. However, if you just post-process the `hyperSpec`{.r} object returned by `read.spc()`{.r}, you should be fine. From 70315bf60248eae81b984883e97145e95c37ce1b Mon Sep 17 00:00:00 2001 From: Erick Oduniyi Date: Wed, 19 Aug 2020 13:19:36 -0500 Subject: [PATCH 06/23] Clean up revisions spotted by @gegznav --- hyperSpec/vignettes/fileio.Rmd | 16 +++++++--------- 1 file changed, 7 insertions(+), 9 deletions(-) diff --git a/hyperSpec/vignettes/fileio.Rmd b/hyperSpec/vignettes/fileio.Rmd index 8a25a8953..2d2fba8c7 100644 --- a/hyperSpec/vignettes/fileio.Rmd +++ b/hyperSpec/vignettes/fileio.Rmd @@ -243,7 +243,7 @@ With the arguments: ---------------- --------------------------------------------------------------------------- -hus, once the data is in R's workspace, creating a `hyperSpec`{.r} object is easy. +Thus, once the data is in R's workspace, creating a `hyperSpec`{.r} object is easy. We suggest wrapping the code to import the data and the line joining it into a `hyperSpec`{.r} object by a user-created import function. Users are more than welcome to contribute such import code to package **hyperSpec**. Secion \@ref(sec:writing-Import) discusses examples of custom import functions. @@ -1198,8 +1198,7 @@ read.txt.Witec.Graph("fileio/txt.Witec/nofilename (Header).txt", encoding = "lat This function reads the spectra files automatically, if they are named properly and extracts additional information of the header file. As for the other Witec functions it is possible to read image data by by selecting `type = "map"`{.r}. Line scans and z-stacks should be read as single spectra. - - unction {#sec:writing-Import} +Function {#sec:writing-Import} This section gives examples of how to write import functions. The first example implements an import filter for an ASCII file format, basically from scratch. The second example shows how to implement @@ -1218,7 +1217,7 @@ more details for an already existing import filter. The raw spectra of the `flu`{.r} data set (see also the separate vignette) are in PerkinElmer's ASCII file format, one spectrum per file. We need a function that automatically reads all files specified by a pattern, such as `*.txt`{.r}. -To gain speed, **hyperSpec** preallocates the spectra matrix after the first reading of the file. +To gain speed, users should preallocate the spectra matrix after the first reading of the file. @@ -1247,8 +1246,7 @@ fluorescence spectra, but it should be possible to change them. - The usual log entry arguments should be supplied. - A sanity check should be implemented: stop with an error if a file does not have the same wavelength axis as the others. -- If no file can be found, an empty `hyperSpec`{.r} object is a consistent result: There is no need -to stop with an error, but it is polite to issue an additional warning. +- If no file can be found, an empty `hyperSpec`{.r} object is a consistent result: There is no need to stop with an error, but it is polite to issue an additional warning. - Finally, package **hyperSpec** does some optional common post-processing for all imported files such as attaching the filename (or connection description) to the `hyperSpec`{.r} object (column `filename`{.r}) and deleting empty spectra. These options can be globally switched on or off by options. @@ -1261,7 +1259,7 @@ writeLines(readLines("read.txt.PerkinElmer.R")) imports the spectra. Note how the labels are set. -The label with the unique name `.wavelength`{.r} corresponds to the wavelength axis, all data columns should have a label with the same name. +The label with the special name `.wavelength`{.r} corresponds to the wavelength axis, all data columns should have a label with the same name. The spectra are always in a data column called `spc`{.r}. Thus, @@ -1296,7 +1294,7 @@ The units of the spectra are saved as well. Function `read.ENVI.Nicolet()`{.r} thus first adjusts the parameters for `read.ENVI()`{.r}. Then `read.ENVI()`{.r} does the main work of importing the file. -The resulting `hyperSpec`{.r} object is post-processed according to the unique header entries. +The resulting `hyperSpec`{.r} object is post-processed according to the special header entries. For using the function, see section \@ref(sec:read-ENVI-Nicolet). @@ -1309,7 +1307,7 @@ writeLines(readLines("read.ENVI.Nicolet.R")) ## Deriving import filters for `spc` files -Please note that future changes inside the `read.spc()`{.r} function is likely to occur. +Please note that future changes inside the `read.spc()`{.r} function are likely to occur. However, if you just post-process the `hyperSpec`{.r} object returned by `read.spc()`{.r}, you should be fine. From 8103373430b371bfa142c87a77d7ac21aae4ae59 Mon Sep 17 00:00:00 2001 From: Erick Oduniyi Date: Wed, 19 Aug 2020 16:57:36 -0500 Subject: [PATCH 07/23] Add header back to correct information --- hyperSpec/vignettes/fileio.Rmd | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hyperSpec/vignettes/fileio.Rmd b/hyperSpec/vignettes/fileio.Rmd index 2d2fba8c7..eeb5a0346 100644 --- a/hyperSpec/vignettes/fileio.Rmd +++ b/hyperSpec/vignettes/fileio.Rmd @@ -1198,7 +1198,7 @@ read.txt.Witec.Graph("fileio/txt.Witec/nofilename (Header).txt", encoding = "lat This function reads the spectra files automatically, if they are named properly and extracts additional information of the header file. As for the other Witec functions it is possible to read image data by by selecting `type = "map"`{.r}. Line scans and z-stacks should be read as single spectra. -Function {#sec:writing-Import} +# Writing your own Import Function {#sec:writing-Import} This section gives examples of how to write import functions. The first example implements an import filter for an ASCII file format, basically from scratch. The second example shows how to implement From 3d880f64908c14086cec4c50f134e6055864c374 Mon Sep 17 00:00:00 2001 From: Erick Oduniyi Date: Wed, 19 Aug 2020 17:05:39 -0500 Subject: [PATCH 08/23] Revise "## Witec {#sec:read-txt-Witec}" section --- hyperSpec/vignettes/fileio.Rmd | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/hyperSpec/vignettes/fileio.Rmd b/hyperSpec/vignettes/fileio.Rmd index eeb5a0346..c7af03f51 100644 --- a/hyperSpec/vignettes/fileio.Rmd +++ b/hyperSpec/vignettes/fileio.Rmd @@ -1099,7 +1099,7 @@ read.spc("fileio/spc.Witec/Witec-Map.spc") ``` `.spc` is in general the recommended format for package **hyperSpec** import. -For imaging data no spatial information for the set of spectra is provided (in version 2.10 this export option is not supported). +For imaging data, no spatial information for the set of spectra is provided (in version 2.10, this export option is not supported). Imaging data (but also single spectra and time series) can be exported as ASCII X and Y files (Save ASCII X and Save ASCII Y, not supported in version 4). These can be read by `read.dat.Witec()`{.r}: @@ -1112,20 +1112,20 @@ read.dat.Witec( ) ``` -Note that the Y data files also contain a wavelength information, but (at least Witec Project 2.10) this information is always wavelength in nm, not Raman shift in wavenumbers: this is provided by the X data file only. +Note that the Y data files also contain wavelength information, but (at least Witec Project 2.10) this information is always wavelength in nm, not Raman shift in wavenumbers: this is provided by the X data file only. Another option is Witec's txt table ASCII export (Export $\rightarrow$ Table), which produces ASCII files with each row corresponding to one wavelength. -The first column contains the wavelength axis, all further columns contain one spectrum each column. +The first column contains the wavelength axis; all further columns contain one spectrum each column. Such files can be read with `read.txt.Witec()`{.r}: ```{r witec-txt, include=FALSE} read.txt.Witec("fileio/txt.Witec/Witec-timeseries_no.txt") ``` -`read.txt.Witec()`{.r} determines the number of wavelengths automaticallly. +`read.txt.Witec()`{.r} determines the number of wavelengths automatically. Note that there are several Export Filter Options. -Here you can determine, which units should be used for the export (see XUnits tab). -In addition, it is possible to export two additional header lines containing information about spectra labels and units. +It is possible to determine which units should be used for the export (see XUnits tab). +It is also possible to export two additional header lines containing information about spectra labels and units. Therefore parameters `hdr.label`{.r} and `hdr.units`{.r} have to be set properly. Otherwise, either an error will be displayed like @@ -1186,9 +1186,9 @@ read.txt.Witec("fileio/txt.Witec/Witec-Map_no.txt", ) ``` -For line scans and z-stacks use `type = "single"`{.r} because the provided information are looking the same like for timeseries, so no further information can be extracted from the header files. +For line scans and z-stacks use `type = "single"`{.r} because the provided information is looking the same as for time series, so no further information can be extracted from the header files. -Since version 4 WITec Project offers the Graph ASCII export (Export $\rightarrow$ Graph ASCII) which produces three ASCII files, named Header containing additional information, X-Axis containing the wavelength values and Y-Axis containing the spectra one spectrum in each column. Data exported in this way can be read with `read.txt.Witec.Graph()`{.r}: +Since version 4 WITec Project offers the Graph ASCII export (Export $\rightarrow$ Graph ASCII), which produces three ASCII files, named Header containing additional information, X-Axis containing the wavelength values and Y-Axis containing the spectra one spectrum in each column. Data exported in this way can be read with `read.txt.Witec.Graph()`{.r}: ```{r witec-txt-Graph, results='hide'} read.txt.Witec.Graph("fileio/txt.Witec/Witec-timeseries (Header).txt") @@ -1196,7 +1196,7 @@ read.txt.Witec.Graph("fileio/txt.Witec/Witec-Map (Header).txt", type = "map") read.txt.Witec.Graph("fileio/txt.Witec/nofilename (Header).txt", encoding = "latin1") ``` -This function reads the spectra files automatically, if they are named properly and extracts additional information of the header file. As for the other Witec functions it is possible to read image data by by selecting `type = "map"`{.r}. Line scans and z-stacks should be read as single spectra. +This function reads the spectra files automatically, if they are appropriately named and extracts additional information of the header file. As for the other Witec functions, it is possible to read image data by selecting `type = "map"`{.r}. Line scans and z-stacks should be read as single spectra. # Writing your own Import Function {#sec:writing-Import} From 55616d3770d4efc35b521555969f89a9c49c375d Mon Sep 17 00:00:00 2001 From: Erick Oduniyi Date: Wed, 19 Aug 2020 17:06:48 -0500 Subject: [PATCH 09/23] Revise "## Horiba / Jobin Yvon (e.g. LabRAM) {#sec:read-txt-Horiba}" section --- hyperSpec/vignettes/fileio.Rmd | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hyperSpec/vignettes/fileio.Rmd b/hyperSpec/vignettes/fileio.Rmd index c7af03f51..dc091243e 100644 --- a/hyperSpec/vignettes/fileio.Rmd +++ b/hyperSpec/vignettes/fileio.Rmd @@ -1036,7 +1036,7 @@ spc <- read.txt.Horiba("fileio/txt.HoribaJobinYvon/ts.txt", spc ``` -Note that Labspec `.txt` files can contains lots of spectra with zero intensity: Labspec saves a complete rectangular grid even if only part of a map was measured. +Note that Labspec `.txt` files can contain lots of spectra with zero intensity: Labspec saves a complete rectangular grid even if only part of a map was measured. These spectra are by removed by default if option `file.remove.emptyspc`{.r} is `TRUE`{.r} (the default). For convenience, functions to further wrappers to import maps (`read.txt.Horiba.xy()`{.r}) and time series (`read.txt.Horiba.t()`{.r}) are provided. From 43aae986b1dcd0428e0f22a6b12f408a4bf64f9d Mon Sep 17 00:00:00 2001 From: Erick Oduniyi Date: Wed, 19 Aug 2020 17:08:09 -0500 Subject: [PATCH 10/23] Revise "### Renishaw ASCII data {#sec:read-txt-Renishaw-ASCII}" section --- hyperSpec/vignettes/fileio.Rmd | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hyperSpec/vignettes/fileio.Rmd b/hyperSpec/vignettes/fileio.Rmd index dc091243e..a90dfed62 100644 --- a/hyperSpec/vignettes/fileio.Rmd +++ b/hyperSpec/vignettes/fileio.Rmd @@ -966,7 +966,7 @@ The ASCII files can easily become very large, particularly with linefocus or str Function `read.txt.Renishaw()`{.r} provides two mechanisms to avoid running out of memory during data import. The file may be imported in chunks of a given number of lines (see the last example). `read.txt.Renishaw()`{.r} can calculate the correct number of wavelengths (i.e., data points per spectrum) if the system command `wc` is available on your computer. -In addition, the processing of the long ASCII format into the spectra matrix is done by reshaping the vector of intensities into a matrix. +Also, the processing of the long ASCII format into the spectra matrix is done by reshaping the vector of intensities into a matrix. This process does not allow any missing values in the data. _Therefore it is not possible to import multi-spectra files with individually "zapped" spectra using `read.txt.Renishaw()`{.r}._ From 0122f317c6585e277553130f6e905c4be64d209e Mon Sep 17 00:00:00 2001 From: Erick Oduniyi Date: Wed, 19 Aug 2020 17:09:22 -0500 Subject: [PATCH 11/23] Revise "## Renishaw Raman {#sec:read-txt-Renishaw}" section --- hyperSpec/vignettes/fileio.Rmd | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/hyperSpec/vignettes/fileio.Rmd b/hyperSpec/vignettes/fileio.Rmd index a90dfed62..fbef71cef 100644 --- a/hyperSpec/vignettes/fileio.Rmd +++ b/hyperSpec/vignettes/fileio.Rmd @@ -940,11 +940,11 @@ spc \index{spc!Renishaw} \index{spc!Raman} -Renishaw's Wire software comes with an file format converter. This program can produce a long ASCII format, `.spc`, or `.jdx` files. +Renishaw's Wire software comes with a file format converter. This program can produce a long ASCII format, `.spc`, or `.jdx` files. -We experienced that the conversion to `.spc` is *not* fully reliable: maps were saved as depth profile, loosing all spatial information. -In addition, an evenly spaced wavelength axis was produced, although this was de-selected in the converter. -We therefore recommend using the ASCII format. +We experienced that the conversion to `.spc` is *not* fully reliable: maps were saved as depth profiles, losing all spatial information. +Also, an evenly spaced wavelength axis was produced, although this was de-selected in the converter. +We, therefore, recommend using the ASCII format. Otherwise the import using `read.spc()`{.r} worked. From b292e3026edacd6afe52451266d1c8d39f83c9f0 Mon Sep 17 00:00:00 2001 From: Erick Oduniyi Date: Wed, 19 Aug 2020 17:19:13 -0500 Subject: [PATCH 12/23] Revise "### Kaiser Optical Systems Raman Maps {#sec:read-spc-KaiserMap}" section --- hyperSpec/vignettes/fileio.Rmd | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/hyperSpec/vignettes/fileio.Rmd b/hyperSpec/vignettes/fileio.Rmd index fbef71cef..f39142bee 100644 --- a/hyperSpec/vignettes/fileio.Rmd +++ b/hyperSpec/vignettes/fileio.Rmd @@ -919,9 +919,9 @@ package **hyperSpec** provides the function `read.spc.KaiserMap()`{.r} to easily `.spc` files written by Kaiser's Hologram software. The filenames of all `.spc` files to be read into one `hyperSpec`{.r} object can be provided either as a character vector or as a wildcard expression (e.g., `"path/to/files/*.spc"`). -The data for the following example was saved with wavelength axis being camera pixels rather than Raman shift. +The data for the following example was saved with the wavelength axis being camera pixels rather than the Raman shift. Thus two files for each spectrum were saved by Hologram. -Thus, a file name pattern is difficult to give and a vector of file names is used instead: +Thus, a file name pattern is difficult to give, and a vector of file names is used instead: ```{r readspcKaiserMap} files <- Sys.glob("fileio/spc.Kaisermap/*.spc") From 900b7dfbcef9f653cd6b40588e19acf08757926f Mon Sep 17 00:00:00 2001 From: Erick Oduniyi Date: Wed, 19 Aug 2020 17:20:54 -0500 Subject: [PATCH 13/23] Revise "## Kaiser Optical Systems Raman {#sec:Kaiser}" section --- hyperSpec/vignettes/fileio.Rmd | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/hyperSpec/vignettes/fileio.Rmd b/hyperSpec/vignettes/fileio.Rmd index f39142bee..73bbf17b0 100644 --- a/hyperSpec/vignettes/fileio.Rmd +++ b/hyperSpec/vignettes/fileio.Rmd @@ -865,7 +865,7 @@ Spectra obtained using Kaiser's Hologram software can be saved either in their o Hologram can also write ASCII files and `.spc` files. We found working with .spc files the best option. -The spectra are usually interpolated by Hologram to an evenly spaced wavelength (or $\Delta\tilde\nu$) axis unless the spectra are saved in a by-pixel manner. +Hologram usually interpolates the spectra to an evenly spaced wavelength (or $\Delta\tilde\nu$) axis unless the spectra are saved in a by-pixel manner. In this case, the full spectra consist of two files with consecutive file names: one for the low and one for the high wavenumber region. See the example for `.spc` import. @@ -882,7 +882,7 @@ The ASCII files are long format that can be imported by `read.txt.long()`{.r} (s We experienced two different problems with these files: -- If the instrument computer's locale is set so that also the decimal separator is a comma, commas are used both as decimal and as column separator. +- If the instrument computer's locale is set so that also the decimal separator is a comma, commas are used both as a decimal and as the column separator. - Values with a decimal fraction of $0$ are written with decimal separator but no further digits (e.g., `2,`). This may be a problem for certain conversion functions (`read.table()`{.r} works fi From 0510eb16ae6c4e08293c7e7c4d99a7090277d1a8 Mon Sep 17 00:00:00 2001 From: Erick Oduniyi Date: Wed, 19 Aug 2020 17:22:20 -0500 Subject: [PATCH 14/23] Revise "## Nicolet FT-IR Imaging {#sec:read-ENVI-Nicolet}" section --- hyperSpec/vignettes/fileio.Rmd | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/hyperSpec/vignettes/fileio.Rmd b/hyperSpec/vignettes/fileio.Rmd index 73bbf17b0..dc3993c9f 100644 --- a/hyperSpec/vignettes/fileio.Rmd +++ b/hyperSpec/vignettes/fileio.Rmd @@ -825,8 +825,8 @@ The proprietary file format of the Opus software is not yet supported. \index{Infrared!ENVI} \index{Nicolet!ENVI} -Also Nicolet saves imaging data in ENVI files. -These files use some non-standard keywords in the header file that should allow to reconstruct the lateral coordinates as well as the wavelength axes and units for wavelength and intensity axis. +Also, Nicolet saves imaging data in ENVI files. +These files use some non-standard keywords in the header file that should reconstruct the lateral coordinates and the wavelength axes and units for wavelength and intensity axis. Package **hyperSpec** has a specialized function `read.ENVI.Nicolet()`{.r} that uses these header entries. It seems that the position of the first spectrum is recorded in $mu m${}, while the pixel size is in mm. From 47e1686bea76a70766143f824c172a6ee3c20163 Mon Sep 17 00:00:00 2001 From: Erick Oduniyi Date: Wed, 19 Aug 2020 17:23:24 -0500 Subject: [PATCH 15/23] Revise "## Bruker FT-IR Imaging {#sec:read-ENVI-Bruker}" section --- hyperSpec/vignettes/fileio.Rmd | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/hyperSpec/vignettes/fileio.Rmd b/hyperSpec/vignettes/fileio.Rmd index dc3993c9f..11d13940f 100644 --- a/hyperSpec/vignettes/fileio.Rmd +++ b/hyperSpec/vignettes/fileio.Rmd @@ -810,8 +810,8 @@ header <- list( ``` No spatial information is given in the ENVI header (if written). -The lateral coordinates can be setup by specifying origin and pixel size for $x$ and $y$ directions. -For details please see the help page. +The lateral coordinates can be set up by specifying origin and pixel size for $x$ and $y$ directions. +For details, please see the help page. The proprietary file format of the Opus software is not yet supported. From fa7fb2140df351e19f3c60db9af7002581b0e7c3 Mon Sep 17 00:00:00 2001 From: Erick Oduniyi Date: Wed, 19 Aug 2020 17:25:13 -0500 Subject: [PATCH 16/23] Revise "## Manufacturer Specific Import Functions {#sec:manuf-spec-import}" section --- hyperSpec/vignettes/fileio.Rmd | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/hyperSpec/vignettes/fileio.Rmd b/hyperSpec/vignettes/fileio.Rmd index 11d13940f..dce51d2a6 100644 --- a/hyperSpec/vignettes/fileio.Rmd +++ b/hyperSpec/vignettes/fileio.Rmd @@ -780,10 +780,10 @@ However, if you just post-process the `hyperSpec`{.r} object returned by `read.s ## Manufacturer Specific Import Functions {#sec:manuf-spec-import} Many spectrometer manufacturers provide a function to export their spectra into ASCII files. -The functions discussed above are written in a very general way, and are highly customizable. -We recommend wrapping these calls with the appropriate settings for your spectra format in an import function. -Please consider contributing such import filters to package **hyperSpec**: send me the documented code (for details see the box at the beginning of this document). -If you are able to import data of any format not mentioned in this document (even without the need of new converters), please let me know (details again in the box at the beginning of this document). +The functions discussed above are written in a very general way and are highly customizable. +We recommend wrapping these calls with the appropriate settings for the spectra format in an import function. +Please consider contributing such import filters to package **hyperSpec**: send us the documented code (for details, see the box at the beginning of this document). +If there is any format not mentioned in this document (even without the need of new converters), please let me know (details again in the box at the beginning of this document). ## Bruker FT-IR Imaging {#sec:read-ENVI-Bruker} From 5e7fbcb041526c4ff3dce4a37ea6fb191270b51e Mon Sep 17 00:00:00 2001 From: Erick Oduniyi Date: Wed, 19 Aug 2020 17:25:55 -0500 Subject: [PATCH 17/23] Revise "### Deriving manufacturer specific import filters" section --- hyperSpec/vignettes/fileio.Rmd | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hyperSpec/vignettes/fileio.Rmd b/hyperSpec/vignettes/fileio.Rmd index dce51d2a6..2ceea241b 100644 --- a/hyperSpec/vignettes/fileio.Rmd +++ b/hyperSpec/vignettes/fileio.Rmd @@ -770,7 +770,7 @@ barbiturates[[, , 25 ~ 30]] ### Deriving manufacturer specific import filters -Please note that future changes inside the read.spc function are likely to occur. +Please note that future changes inside the read.spc function is likely to occur. However, if you just post-process the `hyperSpec`{.r} object returned by `read.spc()`{.r}, you should be fine. From 61dd86479ecbc162d58908782b2fe794018d3d64 Mon Sep 17 00:00:00 2001 From: Erick Oduniyi Date: Wed, 19 Aug 2020 17:28:29 -0500 Subject: [PATCH 18/23] Revise "## spc Files {#sec:read-spc}" section --- hyperSpec/vignettes/fileio.Rmd | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/hyperSpec/vignettes/fileio.Rmd b/hyperSpec/vignettes/fileio.Rmd index 2ceea241b..fef251c74 100644 --- a/hyperSpec/vignettes/fileio.Rmd +++ b/hyperSpec/vignettes/fileio.Rmd @@ -703,14 +703,11 @@ Anyone knowing where it moved please contact me ([**hyperSpec**'s GitHub reposit ``` - - - A variety of sub-formats exists. package **hyperSpec**'s import function `read.spc()`{.r} does *not* support the old file format that was used before 1996. -In addition, no test data with *w planes* was available --- thus the import of such files could not be tested. If you come across such files, please contact the package maintainer ([**hyperSpec**'s GitHub repository](https://github.com/cbeleites/hyperSpec/issues){target="_blank"}). +In addition, no test data with *w planes* was available --- thus, the import of such files could not be tested. If you come across such files, please contact the package maintainer ([**hyperSpec**'s GitHub repository](https://github.com/cbeleites/hyperSpec/issues){target="_blank"}). The header and subheader blocks of spc files store additional information of pre-defined types (see the file format specification[@Galactic1997]). -Further information can be stored in the so-called log block at the end of the file, and should be in a key-value format (although even the official example files do not always). +Further information can be stored in the so-called log block at the end of the file and should be in a key-value format (although even the official example files do not always). This information is often useful (Kaiser's Hologram software, e.g., stores the stage position in the log block). Function `read.spc()`{.r} has four arguments that allow fine-grained control of storing such information in the `hyperSpec`{.r} object: @@ -728,7 +725,7 @@ Function `read.spc()`{.r} has four arguments that allow fine-grained control of The value of these arguments can either be logical (amounting to either use all or none of the information in the file) or a character vector giving the names of the parameters that should be used. Note that the header file field names are always lowercase. -Here's how to find out what extra information could be read from the header and log: +Here is how to find out what extra information could be read from the header and log: ```{r} read.spc("fileio/spc.Kaisermap/ebroAVII.spc", keys.hdr2data = TRUE) From 2a9afa5becfa5e50fce8650fceb8a8321c8a5901 Mon Sep 17 00:00:00 2001 From: Erick Oduniyi Date: Wed, 19 Aug 2020 17:29:33 -0500 Subject: [PATCH 19/23] Revise "## ENVI Files {#sec:read-ENVI}" section --- hyperSpec/vignettes/fileio.Rmd | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/hyperSpec/vignettes/fileio.Rmd b/hyperSpec/vignettes/fileio.Rmd index fef251c74..c6d105269 100644 --- a/hyperSpec/vignettes/fileio.Rmd +++ b/hyperSpec/vignettes/fileio.Rmd @@ -642,10 +642,10 @@ Please use `read.mat.Cytospec()`{.r} instead. ENVI files are binary data accompanied by an ASCII header file. Package **hyperSpec**'s function `read.ENVI()`{.r} can be used to import them. -Usually, the header file name is the same as the binary data file name with the suffix replaced by `.hdr`. Otherwise, the header file name can be given via parameter **`headerfile`{.r}**. +Usually, the header file name is the same as the binary data file name, with the suffix replaced by `.hdr`. Otherwise, the header file name can be given via parameter **header file`{.r}**. As we experienced missing header files (Bruker's Opus software frequently produced header files without any content), the data that would usually be read from the header file can also be handed to `read.ENVI()`{.r} as a list in parameter **`header**`{.r}. -Arguments given in `header`{.r} replace corresponding entries of the header file. +Arguments are given in `header`{.r} replace corresponding entries of the header file. The help page gives details on what elements the list should contain, see also the discussion of ENVI files written by Bruker's OPUS software (section \@ref(sec:read-ENVI-Bruker). Here is how to use `read.ENVI()`{.r}: From fd30390fb63d6db1e1144a825b2d6b1bb12d92a4 Mon Sep 17 00:00:00 2001 From: Erick Oduniyi Date: Wed, 19 Aug 2020 17:58:51 -0500 Subject: [PATCH 20/23] Revise "## Matlab Files {#sec:readMat}" section --- hyperSpec/vignettes/fileio.Rmd | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/hyperSpec/vignettes/fileio.Rmd b/hyperSpec/vignettes/fileio.Rmd index c6d105269..4402a1877 100644 --- a/hyperSpec/vignettes/fileio.Rmd +++ b/hyperSpec/vignettes/fileio.Rmd @@ -539,14 +539,12 @@ Further import filters are provided for manufacturer/software specific ASCII for ASCII export can be done in wide and long format using `write.txt.long()`{.r} and `write.txt.wide()`{.r}. If you need a specific header or footer, use R's functions for writing files: `write.table()`{.r}, `write()`{.r}, `cat()`{.r} and so on offer fine-grained control of writing ASCII files. - - # Binary file formats {#sec:binary-file-formats} ## Matlab Files {#sec:readMat} \index{Matlab} -Matlab files can be read and written using the package package **R.matlab**\citep{R.matlab}, which is available at CRAN and can be installed by `install.packages("R.matlab")`{.r}. +Matlab files can be read and written using the package **R.matlab**\citep{R.matlab}, which is available at CRAN and can be installed by `install.packages("R.matlab")`{.r}. ```{r} library(R.matlab) @@ -565,7 +563,7 @@ library(R.matlab) spc.mat <- readMat("fileio/spectra.mat") ``` -If the `.mat` file was saved with compression, the additional package package **Rcompression** is needed. +If the `.mat` file was saved with compression, the additional package **Rcompression** is needed. It can be installed from omegahat: @@ -582,10 +580,10 @@ install.packages("Rcompression", repos = "http://www.omegahat.org/R") See the documentation of package **R.matlab** for more details and possibly needed further packages. Function `readMat()`{.r} imports the `.mat` file's contents as a list. -The variables in the `.mat` file are properly named elements of the list. +The variables in the `.mat` file are appropriately named elements of the list. The `hyperSpec`{.r} object can be created using `new()`{.r}, see section \@ref(sec:new). -Again, you probably want to wrap the import of your matlab files into a function. +Again, users probably want to wrap the import of their Matlab files into a function. ### Matlab Export {#sec:writing-matlab-files} From 82f9858ef7c163e941ec20c1a9eb38a2d80d4a7a Mon Sep 17 00:00:00 2001 From: Erick Oduniyi Date: Wed, 19 Aug 2020 18:09:20 -0500 Subject: [PATCH 21/23] Revise "# ASCII files {#sec:ascii}" section --- hyperSpec/vignettes/fileio.Rmd | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/hyperSpec/vignettes/fileio.Rmd b/hyperSpec/vignettes/fileio.Rmd index 4402a1877..8ec82db63 100644 --- a/hyperSpec/vignettes/fileio.Rmd +++ b/hyperSpec/vignettes/fileio.Rmd @@ -363,9 +363,9 @@ The import functions immediately return a `hyperSpec`{.r} object. Internally, they use `read.table()`{.r}, a very powerful ASCII import function. R supplies another ASCII import function, `scan()`{.r}. Function `scan()`{.r} imports numeric data matrices and is faster than `read.table()`{.r}, but cannot import column names. -If your data does not contain a header or it is not important and can safely be skipped, you may want to import your data using `scan()`{.r}. +If the data does not contain a header or is not important and can safely be skipped, it may want to import the data using `scan()`{.r}. -Note that R allows to use a variety of compressed file formats directly as ASCII files (for example, see section \@ref(sec:read-txt-Renishaw)). +Note that R allows the use a variety of compressed file formats directly as ASCII files (for example, see section \@ref(sec:read-txt-Renishaw)). Also, both `read.txt.long()`{.r} and `read.txt.wide()`{.r} accept connections instead of file names. @@ -389,9 +389,9 @@ Richard Pena asked about importing another ASCII file type: > The first and following columns corresponds to the angle diffraction and the intensity values of samples respectively. -This file thus differs from the ASCII formats discussed above in that the samples are actually in columns whereas `hyperSpec`{.r} expects them to be in rows. +Thus, this file differs from the ASCII formats discussed above in that the samples are actually in columns whereas `hyperSpec`{.r} expects them to be in rows. The header line gives the name of the sample. -Import is straightforward, just the spectra matrix needs to be transposed to make a `hyperSpec`{.r} object: +Import is straightforward, and just the spectra matrix needs to be transposed to make a `hyperSpec`{.r} object: @@ -446,7 +446,7 @@ read.jdx("fileio/jcamp-dx/shimadzu.jdx", encoding = "latin1", keys.hdr2data = TR read.jdx("fileio/jcamp-dx/virgilio.jdx") ``` -The last file has a slight inconsistenty between its meta data and spectroscopic data, causing a message. +The last file has a slight inconsistency between its meta data and spectroscopic data, causing a message. However, the difference is minute compared to the intensities. If this is known in advance, an appropriate tolerance can be chosen: From d35c765ea66bdd7ae318861b5190e4599d213ead Mon Sep 17 00:00:00 2001 From: Vilmantas Gegzna Date: Thu, 20 Aug 2020 13:13:52 +0300 Subject: [PATCH 22/23] Improve one-sentence-per-row --- hyperSpec/vignettes/fileio.Rmd | 23 +++++++++++++++-------- 1 file changed, 15 insertions(+), 8 deletions(-) diff --git a/hyperSpec/vignettes/fileio.Rmd b/hyperSpec/vignettes/fileio.Rmd index 8ec82db63..eb9895ed7 100644 --- a/hyperSpec/vignettes/fileio.Rmd +++ b/hyperSpec/vignettes/fileio.Rmd @@ -111,7 +111,7 @@ knitr::write_bib( **Consistent naming of file import functions in version >= 0.99** From now on, all import functions have names starting with `read.\*()`. -All functions previously named `scan.\*()`. have been renamed accordingly. +All functions previously named `scan.\*()` have been renamed accordingly. ``` @@ -702,7 +702,8 @@ Anyone knowing where it moved please contact me ([**hyperSpec**'s GitHub reposit A variety of sub-formats exists. package **hyperSpec**'s import function `read.spc()`{.r} does *not* support the old file format that was used before 1996. -In addition, no test data with *w planes* was available --- thus, the import of such files could not be tested. If you come across such files, please contact the package maintainer ([**hyperSpec**'s GitHub repository](https://github.com/cbeleites/hyperSpec/issues){target="_blank"}). +In addition, no test data with *w planes* was available --- thus, the import of such files could not be tested. +If you come across such files, please contact the package maintainer ([**hyperSpec**'s GitHub repository](https://github.com/cbeleites/hyperSpec/issues){target="_blank"}). The header and subheader blocks of spc files store additional information of pre-defined types (see the file format specification[@Galactic1997]). Further information can be stored in the so-called log block at the end of the file and should be in a key-value format (although even the official example files do not always). @@ -935,7 +936,8 @@ spc \index{spc!Renishaw} \index{spc!Raman} -Renishaw's Wire software comes with a file format converter. This program can produce a long ASCII format, `.spc`, or `.jdx` files. +Renishaw's Wire software comes with a file format converter. +This program can produce a long ASCII format, `.spc`, or `.jdx` files. We experienced that the conversion to `.spc` is *not* fully reliable: maps were saved as depth profiles, losing all spatial information. Also, an evenly spaced wavelength axis was produced, although this was de-selected in the converter. @@ -959,7 +961,8 @@ The file may be compressed via gzip, bzip2, xz or lzma. Zip compressed files are read via `read.zip.Renishaw()`{.r}. The ASCII files can easily become very large, particularly with linefocus or streamline imaging. Function `read.txt.Renishaw()`{.r} provides two mechanisms to avoid running out of memory during data import. -The file may be imported in chunks of a given number of lines (see the last example). `read.txt.Renishaw()`{.r} can calculate the correct number of wavelengths (i.e., data points per spectrum) if the system command `wc` is available on your computer. +The file may be imported in chunks of a given number of lines (see the last example). +Function `read.txt.Renishaw()`{.r} can calculate the correct number of wavelengths (i.e., data points per spectrum) if the system command `wc` is available on your computer. Also, the processing of the long ASCII format into the spectra matrix is done by reshaping the vector of intensities into a matrix. This process does not allow any missing values in the data. @@ -1183,7 +1186,8 @@ read.txt.Witec("fileio/txt.Witec/Witec-Map_no.txt", For line scans and z-stacks use `type = "single"`{.r} because the provided information is looking the same as for time series, so no further information can be extracted from the header files. -Since version 4 WITec Project offers the Graph ASCII export (Export $\rightarrow$ Graph ASCII), which produces three ASCII files, named Header containing additional information, X-Axis containing the wavelength values and Y-Axis containing the spectra one spectrum in each column. Data exported in this way can be read with `read.txt.Witec.Graph()`{.r}: +Since version 4 WITec Project offers the Graph ASCII export (Export $\rightarrow$ Graph ASCII), which produces three ASCII files, named Header containing additional information, X-Axis containing the wavelength values and Y-Axis containing the spectra one spectrum in each column. +Data exported in this way can be read with `read.txt.Witec.Graph()`{.r}: ```{r witec-txt-Graph, results='hide'} read.txt.Witec.Graph("fileio/txt.Witec/Witec-timeseries (Header).txt") @@ -1191,12 +1195,15 @@ read.txt.Witec.Graph("fileio/txt.Witec/Witec-Map (Header).txt", type = "map") read.txt.Witec.Graph("fileio/txt.Witec/nofilename (Header).txt", encoding = "latin1") ``` -This function reads the spectra files automatically, if they are appropriately named and extracts additional information of the header file. As for the other Witec functions, it is possible to read image data by selecting `type = "map"`{.r}. Line scans and z-stacks should be read as single spectra. +This function reads the spectra files automatically, if they are appropriately named and extracts additional information of the header file. +As for the other Witec functions, it is possible to read image data by selecting `type = "map"`{.r}. +Line scans and z-stacks should be read as single spectra. # Writing your own Import Function {#sec:writing-Import} -This section gives examples of how to write import functions. The first example implements an import -filter for an ASCII file format, basically from scratch. The second example shows how to implement +This section gives examples of how to write import functions. +The first example implements an import filter for an ASCII file format, basically from scratch. +The second example shows how to implement more details for an already existing import filter. From 8d74a6a968a284b9c4de4b4a34f679094dba7596 Mon Sep 17 00:00:00 2001 From: Vilmantas Gegzna Date: Thu, 20 Aug 2020 13:19:15 +0300 Subject: [PATCH 23/23] Add list of vignettes --- hyperSpec/vignettes/fileio.Rmd | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/hyperSpec/vignettes/fileio.Rmd b/hyperSpec/vignettes/fileio.Rmd index eb9895ed7..0f9757f9c 100644 --- a/hyperSpec/vignettes/fileio.Rmd +++ b/hyperSpec/vignettes/fileio.Rmd @@ -170,6 +170,15 @@ This online directory contains the required datasets (via `git-lfs`). *** + + + ```{r, echo = FALSE, results = "asis"} + res <- knitr::knit_child("list-of-vignettes.md", quiet = TRUE) + cat(res, sep = '\n') + ``` + +*** + ```{r} library(hyperSpec) ``` @@ -1216,7 +1225,7 @@ more details for an already existing import filter. \index{PerkinElmer!Fluorescence} \index{PerkinElmer!ASCII long} -The raw spectra of the `flu`{.r} data set (see also the separate vignette) are in PerkinElmer's ASCII file format, one spectrum per file. +The raw spectra of the `flu`{.r} data set (see also the [separate vignette](#list-of-vignettes)) are in PerkinElmer's ASCII file format, one spectrum per file. We need a function that automatically reads all files specified by a pattern, such as `*.txt`{.r}. To gain speed, users should preallocate the spectra matrix after the first reading of the file.