Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(compiler): prevent double full builds #3374

Merged
merged 3 commits into from
May 24, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 15 additions & 2 deletions src/compiler/build/full-build.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,16 @@ import { BuildContext } from './build-ctx';
import { createTsBuildProgram } from '../transpile/create-build-program';
import ts from 'typescript';

export const createFullBuild = async (config: d.Config, compilerCtx: d.CompilerCtx) => {
/**
* Build a callable function to perform a full build of a Stencil project
* @param config a Stencil configuration to apply to a full build of a Stencil project
* @param compilerCtx the current Stencil compiler context
* @returns the results of a full build of Stencil
*/
export const createFullBuild = async (
config: d.Config,
compilerCtx: d.CompilerCtx
): Promise<d.CompilerBuildResults> => {
return new Promise<d.CompilerBuildResults>((resolve) => {
let tsWatchProgram: ts.WatchOfConfigFile<ts.BuilderProgram> = null;

Expand All @@ -13,7 +22,11 @@ export const createFullBuild = async (config: d.Config, compilerCtx: d.CompilerC
compilerCtx.fs.clearFileCache(p);
});

const onBuild = async (tsBuilder: ts.BuilderProgram) => {
/**
* A function that kicks off the transpilation process for both the TypeScript and Stencil compilers
* @param tsBuilder the manager of the {@link ts.Program} state
*/
const onBuild = async (tsBuilder: ts.BuilderProgram): Promise<void> => {
const buildCtx = new BuildContext(config, compilerCtx);
buildCtx.isRebuild = false;
buildCtx.requiresFullBuild = true;
Expand Down
7 changes: 6 additions & 1 deletion src/compiler/compiler.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,12 @@ import { resolveModuleIdAsync } from './sys/resolve/resolve-module-async';
import { isFunction } from '@utils';
import ts from 'typescript';

export const createCompiler = async (config: Config) => {
/**
* Generate a Stencil compiler instance
* @param config a Stencil configuration to apply to the compiler instance
* @returns a new instance of a Stencil compiler
*/
export const createCompiler = async (config: Config): Promise<Compiler> => {
// actual compiler code
// could be in a web worker on the browser
// or the main thread in node
Expand Down
123 changes: 90 additions & 33 deletions src/compiler/transpile/create-build-program.ts
Original file line number Diff line number Diff line change
@@ -1,67 +1,124 @@
import type * as d from '../../declarations';
import { getTsOptionsToExtend } from './ts-config';
import { GENERATED_DTS } from '../output-targets/output-utils';
import ts from 'typescript';

/**
* Create a TypeScript Program ({@link ts.Program}) to perform builds of a Stencil project using the provided
* `buildCallback` entity
* @param config a Stencil configuration to apply to a full build of a Stencil project
* @param buildCallback a callback that invokes the actual transpilation of a Stencil project
* @returns a Program that marries the TypeScript and Stencil compilers together.
*/
export const createTsBuildProgram = async (
config: d.Config,
buildCallback: (tsBuilder: ts.BuilderProgram) => Promise<void>
) => {
let isRunning = false;
let timeoutId: any;
): Promise<ts.WatchOfConfigFile<ts.EmitAndSemanticDiagnosticsBuilderProgram>> => {
let isBuildRunning = false;
let currentBuildTimeoutId: any;

const optionsToExtend = getTsOptionsToExtend(config);

/**
* Create a {@link ts.System}. The System is responsible for handling all interactions between the TypeScript compiler
* and the host operating system.
*/
const tsWatchSys: ts.System = {
...ts.sys,

watchFile(path, callback) {
if (path.endsWith(`/${GENERATED_DTS}`)) {
return ts.sys.watchFile(path, callback);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this basically the offending code here? I was just thinking it might be good to have a brief explanation as to why we have a no-op file watcher here (and under watchDirectory below)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup! That's it.

So admission time - I'm not entirely sure if/why close was overridden the way it was. I followed the pattern to avoid breaking anything further (since this isn't under test ATM). Unfortunately there isn't a design doc or anything in git blame that makes the justification for this immediately obvious 😢

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmmm the mystery deepends...in that case I think it's fine as is 😄

/**
* Watch changes in source files, missing files needed to update the program or config file
* @returns a no-op file watcher
*/
watchFile(): ts.FileWatcher {
return {
close() {},
};
},
watchDirectory() {
/**
* Watch a resolved module's failed lookup locations, config file specs, type roots where auto type reference
* directives are added
* @returns a no-op file watcher
*/
watchDirectory(): ts.FileWatcher {
return {
close() {},
};
},
setTimeout(callback, time) {
timeoutId = setInterval(() => {
if (!isRunning) {
/**
* Set delayed compilation, so that multiple changes in short span are compiled together
* @param callback a callback to invoke upon the completion of compilation. this function is provided to Stencil by
* the TypeScript compiler.
* @param timeoutMs the minimum time to wait (in milliseconds) before checking if compilation is complete or not
* @returns the identifier for the interval that's created
*/
setTimeout(callback: (...args: any[]) => void, timeoutMs: number): any {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we get an error if we change any to number here for the return value?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For both this and clearTimeout, I kept the same function signatures that TypeScript defines in typescript.d.ts:

    export interface System {
        ...
        setTimeout?(callback: (...args: any[]) => void, ms: number, ...args: any[]): any;
        clearTimeout?(timeoutId: any): void;
    }

I think this is due to the fact that in some cases we can use a number, but in Node these functions are typed as accepting NodeJS.Timeout, which is an object that adheres to this interface:

    interface Timeout extends Timer {
        hasRef(): boolean;
        refresh(): this;
        [Symbol.toPrimitive](): number;
    }

    // compatibility with older typings
    interface Timer extends RefCounted {
        hasRef(): boolean;
        refresh(): this;
        [Symbol.toPrimitive](): number;
    }

    interface RefCounted {
        ref(): this;
        unref(): this;
    }

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok yeah that makes sense - I had a feeling this could be a browser <-> node difference

currentBuildTimeoutId = setInterval(() => {
if (!isBuildRunning) {
callback();
clearInterval(timeoutId);
clearInterval(currentBuildTimeoutId);
}
}, config.sys.watchTimeout || time);
return timeoutId;
}, config.sys.watchTimeout || timeoutMs);
return currentBuildTimeoutId;
},

clearTimeout(id) {
return clearInterval(id);
/**
* Reset existing delayed compilation
* @param timeoutId the current build timeout identifier to clear
*/
clearTimeout(timeoutId: any): void {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think timeoutId here can be number too, possibly?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See #3374 (comment) - I think these should be left as any

clearInterval(timeoutId);
},
};

config.sys.addDestory(() => tsWatchSys.clearTimeout(timeoutId));
config.sys.addDestory(() => tsWatchSys.clearTimeout(currentBuildTimeoutId));

const tsWatchHost = ts.createWatchCompilerHost(
config.tsconfig,
optionsToExtend,
tsWatchSys,
ts.createEmitAndSemanticDiagnosticsBuilderProgram,
(reportDiagnostic) => {
config.logger.debug('watch reportDiagnostic:' + reportDiagnostic.messageText);
},
(reportWatchStatus) => {
config.logger.debug(reportWatchStatus.messageText);
}
);
/**
* Create a {@link ts.WatchCompilerHost}. A CompilerHost allows a {@link ts.Program} to interact with the
* {@link ts.System}, by acting as an intermediary:
* ```
* ┌────────────┐ ┌──────────────────────┐ ┌───────────┐ ┌──────────────────┐
* │ ts.Program │<->│ ts.WatchCompilerHost │<->│ ts.System │<->│ Operating System │
* └────────────┘ └──────────────────────┘ └───────────┘ └──────────────────┘
* ```
*
* Strictly speaking, the created entity is a subclass of a WatchCompilerHost. The
* {@link ts.WatchCompilerHostOfConfigFile} class has the following features that makes it useful to Stencil (even
* when Stencil is performing a single, full build):
* - it provides the opportunity to extend/alter an existing tsconfig file, allowing users to override specific
* configuration options via {@link ts.WatchCompilerHostOfConfigFile#optionsToExtend}, which is a provided as an
* argument in the constructor
* - it includes the {@link ts.WatchCompilerHost#afterProgramCreate} function in its interface, which Stencil
* overrides to invoke a build callback (not as a part of this object's creation)
*/
const tsWatchHost: ts.WatchCompilerHostOfConfigFile<ts.EmitAndSemanticDiagnosticsBuilderProgram> =
ts.createWatchCompilerHost(
config.tsconfig,
optionsToExtend,
tsWatchSys,
ts.createEmitAndSemanticDiagnosticsBuilderProgram,
(reportDiagnostic) => {
config.logger.debug('watch reportDiagnostic:' + reportDiagnostic.messageText);
},
(reportWatchStatus) => {
config.logger.debug(reportWatchStatus.messageText);
}
);

tsWatchHost.afterProgramCreate = async (tsBuilder) => {
isRunning = true;
/**
* Override {@link ts.WatchCompilerHost#afterProgramCreate} to invoke the build callback that was provided as an
* argument to this function.
* @param tsBuilder a {@link ts.BuilderProgram} to manage the {@link ts.Program} in the provided build context
*/
tsWatchHost.afterProgramCreate = async (tsBuilder: ts.EmitAndSemanticDiagnosticsBuilderProgram): Promise<void> => {
isBuildRunning = true;
await buildCallback(tsBuilder);
isRunning = false;
isBuildRunning = false;
};

/**
* Create the initial {@link ts.Program} using Stencil's custom {@link ts.WatchCompilerHostOfConfigFile}. The Program
* represents the _TypeScript_ compiler context, that will work in tandem with Stencil's compiler context and build
* context
*/
return ts.createWatchProgram(tsWatchHost);
};
2 changes: 1 addition & 1 deletion src/compiler/types/generate-app-types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ export const generateAppTypes = async (
};

/**
* Generates a `component.d.ts` file's contents, which contains the typings for all components in a Stencil project
* Generates a `components.d.ts` file's contents, which contains the typings for all components in a Stencil project
* @param config the Stencil configuration associated with the project being compiled
* @param buildCtx the context associated with the current build
* @param areTypesInternal determines if non-exported type definitions are being generated or not
Expand Down