Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Choice of Hash Function #29

Merged
merged 28 commits into from
Jun 26, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
7176cfc
update refs ← package.json
philcockfield Jun 23, 2023
87e1e3e
refactor: isolate types
philcockfield Jun 23, 2023
53aeea0
Add "algorithm" params (current "sha1")
philcockfield Jun 23, 2023
925b971
comment
philcockfield Jun 23, 2023
c68b06b
consolidate util tests
philcockfield Jun 23, 2023
b636998
Add "sha256" option ← tests for util hashing function
philcockfield Jun 23, 2023
1ff1094
refactor
philcockfield Jun 23, 2023
720a752
weave in tests for SHA256 to root level cache
philcockfield Jun 23, 2023
ceb6e1b
Add algorithm: "sha512"
philcockfield Jun 23, 2023
522cb3d
Update README.md
philcockfield Jun 23, 2023
e222909
readme
philcockfield Jun 23, 2023
63a20ec
Update README.md
philcockfield Jun 23, 2023
4c05c29
Update README.md
philcockfield Jun 23, 2023
318933f
Update README.md
philcockfield Jun 23, 2023
27e5272
tmp rename
philcockfield Jun 26, 2023
502c29b
rename (lowercase)
philcockfield Jun 26, 2023
28a414a
add dev-dependency (ts-node)
philcockfield Jun 26, 2023
90e7d17
refactor
philcockfield Jun 26, 2023
53e7058
hash union type generator
philcockfield Jun 26, 2023
b583e3f
refactor
philcockfield Jun 26, 2023
984e029
generated hashses: union type AND constant
philcockfield Jun 26, 2023
49ec97c
expose static constant
philcockfield Jun 26, 2023
07619c5
tests for updated hashes
philcockfield Jun 26, 2023
a1b53b0
Update CHANGELOG.md
philcockfield Jun 26, 2023
ce10dff
Update util.TEST.ts
philcockfield Jun 26, 2023
432ccba
refactor
philcockfield Jun 26, 2023
b889ae0
ensure hash exists at construction
philcockfield Jun 26, 2023
11c9ecf
2.4.0
philcockfield Jun 26, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,11 @@ This project adheres to [Semantic Versioning](http://semver.org/).
#### Security


## [2.4.0] - 2023-06-27
#### Added
- multiple path hashing options (thanks @trevor-vaughan)


## [2.3.0] - 2023-05-22
#### Added
- `ttl` (time to live) to expire caches (thanks @douglaslinsmeyer)
Expand Down
17 changes: 14 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,16 @@ A super-fast, promise-based cache that reads and writes to the file-system.

npm install --save file-system-cache

Import

```ts
import Cache from 'file-system-cache';

// or ↑↓ (equivalent)

import { Cache } from 'file-system-cache';
```

## Usage (API)

Create an instance of the cache optionally giving it a folder location to store files within.
Expand All @@ -17,9 +27,10 @@ Create an instance of the cache optionally giving it a folder location to store
import Cache from "file-system-cache";

const cache = Cache({
basePath: "./.cache", // Optional. Path where cache files are stored (default).
ns: "my-namespace", // Optional. A grouping namespace for items.
ttl: 60 // Optional. A time-to-live for how long the cached item is valid (secs).
basePath: "./.cache", // (optional) Path where cache files are stored (default).
ns: "my-namespace", // (optional) A grouping namespace for items.
hash: "sha1" // (optional) A hashing algorithm used within the cache key.
ttl: 60 // (optional) A time-to-live (in secs) on how long an item remains cached.
});
```

Expand Down
12 changes: 7 additions & 5 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,27 +1,29 @@
{
"name": "file-system-cache",
"version": "2.3.0",
"version": "2.4.0",
"description": "A super-fast, promise-based cache that reads and writes to the file-system.",
"main": "./lib/index.js",
"scripts": {
"test": "ts-mocha -p tsconfig.json src/**/*.TEST.ts",
"build": "tsc",
"prepublish": "npm test && npm run build"
"prepublish": "npm test && npm run build",
"gen:hashtype": "ts-node-esm -T ./script.ts/generate-hashtype.ts"
},
"dependencies": {
"fs-extra": "11.1.1",
"ramda": "0.29.0"
},
"devDependencies": {
"@types/chai": "4.3.4",
"@types/chai": "4.3.5",
"@types/expect": "24.3.0",
"@types/fs-extra": "11.0.1",
"@types/mocha": "10.0.1",
"@types/ramda": "0.29.1",
"@types/ramda": "0.29.2",
"chai": "4.3.7",
"mocha": "10.2.0",
"ts-mocha": "10.0.0",
"typescript": "5.0.4"
"ts-node": "10.9.1",
"typescript": "5.1.3"
},
"repository": {
"type": "git",
Expand Down
45 changes: 45 additions & 0 deletions script.ts/generate-hashtype.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
#!/usr/bin/env ts-node
import { crypto, fs, fsPath } from '../src/common';

const HashUnionType = {
/**
* Generate the union type.
*/
generate() {
const names = crypto.getHashes().map((hash) => `'${hash}'`);
const type = `export type HashAlgorithm = \n | ${names.join('\n | ')};`;

const nameArray = `[\n ${names.join(',\n ')}];`;
const constants = `export const hashAlgorithms: t.HashAlgorithm[] = ${nameArray}`;
return { type, constants };
},

/**
* Generate and save the type definition file.
*/
update() {
const header = `
/**
* Generated file - do not edit.
* See:
* - generator: script.ts/generate-hashtype.ts
* - command: yarn run gen:hashtype
*/
`
.substring(1)
.slice(0, -1);

const { type, constants } = HashUnionType.generate();
const typeDef = `${header}\n${type}\n`;
const importT = `import { type t } from '../common.t';`;
const constDef = `${header}\n${importT}\n\n${constants}\n`;

fs.writeFileSync(fsPath.resolve('./src/types.hashes.ts'), typeDef);
fs.writeFileSync(fsPath.resolve('./src/common/const.hashes.ts'), constDef);
},
};

/**
* Run
*/
HashUnionType.update();
42 changes: 26 additions & 16 deletions src/FileSystemCache.ts
Original file line number Diff line number Diff line change
@@ -1,21 +1,23 @@
import { R, fs, Util } from './common';

export type FileSystemCacheOptions = {
basePath?: string;
ns?: any;
extension?: string;
ttl?: number;
};
import { R, Util, fs, hashAlgorithms, type t } from './common';

/**
* A cache that read/writes to a specific part of the file-system.
*/
export class FileSystemCache {
basePath: string;
ns?: any;
extension?: string;
/**
* The list of all available hash algorithms.
*/
static hashAlgorithms: t.HashAlgorithm[] = hashAlgorithms;

/**
* Instance.
*/
readonly basePath: string;
readonly ns?: any;
readonly extension?: string;
readonly hash: t.HashAlgorithm;
readonly ttl: number;
basePathExists?: boolean;
ttl: number;

/**
* Constructor.
Expand All @@ -28,15 +30,23 @@ export class FileSystemCache {
* - extension: An optional file-extension for paths.
* - ttl: The default time-to-live for cached values in seconds.
* Default: 0 (never expires)
* - hash: The hashing algorithm to use when generating cache keys.
* Default: "sha1"
*/
constructor(options: FileSystemCacheOptions = {}) {
constructor(options: t.FileSystemCacheOptions = {}) {
this.basePath = formatPath(options.basePath);
this.ns = Util.hash(options.ns);
this.ttl = typeof options.ttl === 'undefined' ? 0 : options.ttl;
this.hash = options.hash ?? 'sha1';
this.ns = Util.hash(this.hash, options.ns);
this.ttl = options.ttl ?? 0;
if (Util.isString(options.extension)) this.extension = options.extension;

if (Util.isFileSync(this.basePath)) {
throw new Error(`The basePath '${this.basePath}' is a file. It should be a folder.`);
}

if (!Util.hashExists(this.hash)) {
throw new Error(`Hash does not exist: ${this.hash}`);
}
}

/**
Expand All @@ -45,7 +55,7 @@ export class FileSystemCache {
*/
public path(key: string): string {
if (Util.isNothing(key)) throw new Error(`Path requires a cache key.`);
let name = Util.hash(key);
let name = Util.hash(this.hash, key);
if (this.ns) name = `${this.ns}-${name}`;
if (this.extension) name = `${name}.${this.extension.replace(/^\./, '')}`;
return `${this.basePath}/${name}`;
Expand Down
2 changes: 2 additions & 0 deletions src/common.t.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
import type * as t from './types';
export type { t };
61 changes: 61 additions & 0 deletions src/common/const.hashes.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
/**
* Generated file - do not edit.
* See:
* - generator: script.ts/generate-hashtype.ts
* - command: yarn run gen:hashtype
*/
import { type t } from '../common.t';

export const hashAlgorithms: t.HashAlgorithm[] = [
'RSA-MD5',
'RSA-RIPEMD160',
'RSA-SHA1',
'RSA-SHA1-2',
'RSA-SHA224',
'RSA-SHA256',
'RSA-SHA3-224',
'RSA-SHA3-256',
'RSA-SHA3-384',
'RSA-SHA3-512',
'RSA-SHA384',
'RSA-SHA512',
'RSA-SHA512/224',
'RSA-SHA512/256',
'RSA-SM3',
'blake2b512',
'blake2s256',
'id-rsassa-pkcs1-v1_5-with-sha3-224',
'id-rsassa-pkcs1-v1_5-with-sha3-256',
'id-rsassa-pkcs1-v1_5-with-sha3-384',
'id-rsassa-pkcs1-v1_5-with-sha3-512',
'md5',
'md5-sha1',
'md5WithRSAEncryption',
'ripemd',
'ripemd160',
'ripemd160WithRSA',
'rmd160',
'sha1',
'sha1WithRSAEncryption',
'sha224',
'sha224WithRSAEncryption',
'sha256',
'sha256WithRSAEncryption',
'sha3-224',
'sha3-256',
'sha3-384',
'sha3-512',
'sha384',
'sha384WithRSAEncryption',
'sha512',
'sha512-224',
'sha512-224WithRSAEncryption',
'sha512-256',
'sha512-256WithRSAEncryption',
'sha512WithRSAEncryption',
'shake128',
'shake256',
'sm3',
'sm3WithRSAEncryption',
'ssl3-md5',
'ssl3-sha1'];
7 changes: 5 additions & 2 deletions src/common/index.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
import * as Util from './Util';
import * as Util from './util';

export { Util };
export * from './const.hashes';
export * from './libs';
export { Util };

export { type t } from '../common.t';
6 changes: 3 additions & 3 deletions src/common/libs.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import R from 'ramda';
import crypto from 'crypto';
import fs from 'fs-extra';
import fsPath from 'path';
import crypto from 'crypto';
import R from 'ramda';

export { R, fs, fsPath, crypto };
export { R, crypto, fs, fsPath };
11 changes: 8 additions & 3 deletions src/common/Util.ts → src/common/util.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import { R, fs, fsPath, crypto } from './libs';
import { type t } from '../common.t';
import { R, crypto, fs, fsPath } from './libs';

export const isNothing = (value: any) => R.isNil(value) || R.isEmpty(value);
export const isString = R.is(String);
Expand Down Expand Up @@ -39,15 +40,19 @@ export const filePathsP = async (basePath: string, ns: string): Promise<string[]
* Turns a set of values into a HEX hash code.
* @param values: The set of values to hash.
*/
export const hash = (...values: any[]) => {
export const hash = (algorithm: t.HashAlgorithm, ...values: any[]) => {
if (R.pipe(compact, R.isEmpty)(values)) return undefined;
const resultHash = crypto.createHash('sha1');
const resultHash = crypto.createHash(algorithm);
const addValue = (value: any) => resultHash.update(value);
const addValues = R.forEach(addValue);
R.pipe(toStringArray, addValues)(values);
return resultHash.digest('hex');
};

export const hashExists = (algorithm: t.HashAlgorithm) => {
return crypto.getHashes().includes(algorithm);
};

/**
* Retrieve a value from the given path.
*/
Expand Down
7 changes: 4 additions & 3 deletions src/index.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
import { FileSystemCache, FileSystemCacheOptions } from './FileSystemCache';
import { type t } from './common';
import { FileSystemCache } from './FileSystemCache';

/**
* Default entry function.
*/
export default (options?: FileSystemCacheOptions) => new FileSystemCache(options);
export { FileSystemCache };
export default (options?: t.FileSystemCacheOptions) => new FileSystemCache(options);
export { FileSystemCache, FileSystemCache as Cache };
Loading