Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(lambda): Connect CW Live Tail to Lambda functions #6423

Merged
merged 2 commits into from
Jan 28, 2025
Merged

feat(lambda): Connect CW Live Tail to Lambda functions #6423

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
8d48a3c
feat(lambda): Connect CW Live Tail to Lambda functions
valerena Nov 21, 2024
cfacb9a
Update wording for changeset
valerena Jan 23, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions packages/core/package.nls.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,7 @@
"AWS.command.appBuilder.deploy": "Deploy SAM Application",
"AWS.command.appBuilder.build": "Build SAM Template",
"AWS.command.appBuilder.searchLogs": "Search Logs",
"AWS.command.appBuilder.tailLogs": "Tail Logs",
Copy link
Contributor

@justinmk3 justinmk3 Jan 23, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably use the same wording (and actually the same AWS.foo.bar id, instead of multiple) in all places:

aws-toolkit-vscode/packages/core/package.nls.json

Lines 181 to 182 in 8d48a3c

"AWS.command.cloudWatchLogs.searchLogGroup": "Search Log Group",
"AWS.command.cloudWatchLogs.tailLogGroup": "Tail Log Group",

However, not a blocker for this PR. Needs a decision about which wording to use in both places.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The difference here is that the one you linked shows "when you have a log group selected" (therefore the wording is "tail log group" as in "tail this log group"), while this new one is when you have a function selected. Maybe this could be "Tail function logs" instead of just "Tail logs". The inspiration was the other existing string right above "Search Logs" (with its corresponding "Search Log Group"), which is a similar behavior, but that starts a search instead of using live tail.

Let me know if this difference makes sense or if we should still try to unify the wording.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The inspiration was the other existing string right above "Search Logs"

Yup I noticed that too, and I like the brevity of "Logs" instead of "Log Group". For now let's leave it as you have in this PR.

"AWS.command.refreshappBuilderExplorer": "Refresh Application Builder Explorer",
"AWS.command.applicationComposer.openDialog": "Open Template with Infrastructure Composer...",
"AWS.command.auth.addConnection": "Add New Connection",
Expand Down
29 changes: 0 additions & 29 deletions packages/core/src/awsService/appBuilder/explorer/nodes/deployedNode.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@ import { createPlaceholderItem } from '../../../../shared/treeview/utils'
import * as nls from 'vscode-nls'

import { getLogger } from '../../../../shared/logger/logger'
import { FunctionConfiguration, LambdaClient, GetFunctionCommand } from '@aws-sdk/client-lambda'
import { DefaultLambdaClient } from '../../../../shared/clients/lambdaClient'
import globals from '../../../../shared/extensionGlobals'
import { defaultPartition } from '../../../../shared/regions/regionProvider'
Expand All @@ -28,7 +27,6 @@ import {
s3BucketType,
} from '../../../../shared/cloudformation/cloudformation'
import { ToolkitError } from '../../../../shared'
import { getIAMConnection } from '../../../../auth/utils'

const localize = nls.loadMessageBundle()
export interface DeployedResource {
Expand Down Expand Up @@ -89,8 +87,6 @@ export async function generateDeployedNode(
const defaultClient = new DefaultLambdaClient(regionCode)
const lambdaNode = new LambdaNode(regionCode, defaultClient)
let configuration: Lambda.FunctionConfiguration
let v3configuration
let logGroupName
try {
configuration = (await defaultClient.getFunction(deployedResource.PhysicalResourceId))
.Configuration as Lambda.FunctionConfiguration
Expand All @@ -101,31 +97,6 @@ export async function generateDeployedNode(
code: 'lambdaClientError',
})
}
const connection = await getIAMConnection({ prompt: false })
if (!connection || connection.type !== 'iam') {
return [
createPlaceholderItem(
localize(
'AWS.appBuilder.explorerNode.unavailableDeployedResource',
'[Failed to retrive deployed resource. Ensure your AWS account is connected.]'
)
),
]
}
const cred = await connection.getCredentials()
const v3Client = new LambdaClient({ region: regionCode, credentials: cred })

const v3command = new GetFunctionCommand({ FunctionName: deployedResource.PhysicalResourceId })
try {
v3configuration = (await v3Client.send(v3command)).Configuration as FunctionConfiguration
logGroupName = v3configuration.LoggingConfig?.LogGroup
} catch (error: any) {
getLogger().error('Error getting Lambda V3 configuration: %O', error)
}
newDeployedResource.configuration = {
...newDeployedResource.configuration,
logGroupName: logGroupName,
} as any
break
}
case s3BucketType: {
Expand Down
8 changes: 4 additions & 4 deletions packages/core/src/awsService/cloudWatchLogs/activation.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,13 +28,13 @@ import { getLogger } from '../../shared/logger/logger'
import { ToolkitError } from '../../shared'
import { LiveTailCodeLensProvider } from './document/liveTailCodeLensProvider'

export const liveTailRegistry = LiveTailSessionRegistry.instance
export const liveTailCodeLensProvider = new LiveTailCodeLensProvider(liveTailRegistry)
export async function activate(context: vscode.ExtensionContext, configuration: Settings): Promise<void> {
const registry = LogDataRegistry.instance
const liveTailRegistry = LiveTailSessionRegistry.instance

const documentProvider = new LogDataDocumentProvider(registry)
const liveTailDocumentProvider = new LiveTailDocumentProvider()
const liveTailCodeLensProvider = new LiveTailCodeLensProvider(liveTailRegistry)
context.subscriptions.push(
vscode.languages.registerCodeLensProvider(
{
Expand Down Expand Up @@ -150,7 +150,7 @@ export async function activate(context: vscode.ExtensionContext, configuration:
)
}

function getFunctionLogGroupName(configuration: any) {
export function getFunctionLogGroupName(configuration: any) {
const logGroupPrefix = '/aws/lambda/'
return configuration.logGroupName || logGroupPrefix + configuration.FunctionName
return configuration.LoggingConfig?.LogGroup || logGroupPrefix + configuration.FunctionName
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this because SDK update?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's related. The code that was removed in the other file used to set up this logGroupName variable that wasn't in the original configuration, but here we're just taking it directly from the Lambda Function configuration object that comes from the SDK.

}
6 changes: 4 additions & 2 deletions packages/core/src/awsService/cloudWatchLogs/commands/tailLogGroup.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,15 +17,17 @@ import {
import { getLogger, globals, ToolkitError } from '../../../shared'
import { uriToKey } from '../cloudWatchLogsUtils'
import { LiveTailCodeLensProvider } from '../document/liveTailCodeLensProvider'
import { LogStreamFilterResponse } from '../wizard/liveTailLogStreamSubmenu'

export async function tailLogGroup(
registry: LiveTailSessionRegistry,
source: string,
codeLensProvider: LiveTailCodeLensProvider,
logData?: { regionName: string; groupName: string }
logData?: { regionName: string; groupName: string },
logStreamFilterData?: LogStreamFilterResponse
): Promise<void> {
await telemetry.cloudwatchlogs_startLiveTail.run(async (span) => {
const wizard = new TailLogGroupWizard(logData)
const wizard = new TailLogGroupWizard(logData, logStreamFilterData)
const wizardResponse = await wizard.run()
if (!wizardResponse) {
throw new CancellationError('user')
Expand Down
3 changes: 2 additions & 1 deletion packages/core/src/awsService/cloudWatchLogs/wizard/tailLogGroupWizard.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ export interface TailLogGroupWizardResponse {
}

export class TailLogGroupWizard extends Wizard<TailLogGroupWizardResponse> {
public constructor(logGroupInfo?: CloudWatchLogsGroupInfo) {
public constructor(logGroupInfo?: CloudWatchLogsGroupInfo, logStreamInfo?: LogStreamFilterResponse) {
super({
initState: {
regionLogGroupSubmenuResponse: logGroupInfo
Expand All @@ -33,6 +33,7 @@ export class TailLogGroupWizard extends Wizard<TailLogGroupWizardResponse> {
region: logGroupInfo.regionName,
}
: undefined,
logStreamFilter: logStreamInfo ?? undefined,
},
})
this.form.regionLogGroupSubmenuResponse.bindPrompter(createRegionLogGroupSubmenu)
Expand Down
42 changes: 41 additions & 1 deletion packages/core/src/lambda/activation.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
*/

import * as vscode from 'vscode'
import { Lambda } from 'aws-sdk'
import { deleteLambda } from './commands/deleteLambda'
import { uploadLambdaCommand } from './commands/uploadLambda'
import { LambdaFunctionNode } from './explorer/lambdaFunctionNode'
Expand All @@ -18,6 +19,11 @@ import { copyLambdaUrl } from './commands/copyLambdaUrl'
import { ResourceNode } from '../awsService/appBuilder/explorer/nodes/resourceNode'
import { isTreeNode, TreeNode } from '../shared/treeview/resourceTreeDataProvider'
import { getSourceNode } from '../shared/utilities/treeNodeUtils'
import { tailLogGroup } from '../awsService/cloudWatchLogs/commands/tailLogGroup'
import { liveTailRegistry, liveTailCodeLensProvider } from '../awsService/cloudWatchLogs/activation'
import { getFunctionLogGroupName } from '../awsService/cloudWatchLogs/activation'
import { ToolkitError, isError } from '../shared'
import { LogStreamFilterResponse } from '../awsService/cloudWatchLogs/wizard/liveTailLogStreamSubmenu'

/**
* Activates Lambda components.
Expand Down Expand Up @@ -76,6 +82,40 @@ export async function activate(context: ExtContext): Promise<void> {

Commands.register('aws.launchDebugConfigForm', async (node: ResourceNode) =>
registerSamDebugInvokeVueCommand(context.extensionContext, { resource: node })
)
),

Commands.register('aws.appBuilder.tailLogs', async (node: LambdaFunctionNode | TreeNode) => {
let functionConfiguration: Lambda.FunctionConfiguration
try {
const sourceNode = getSourceNode<LambdaFunctionNode>(node)
functionConfiguration = sourceNode.configuration
const logGroupInfo = {
regionName: sourceNode.regionCode,
groupName: getFunctionLogGroupName(functionConfiguration),
}

const source = isTreeNode(node) ? 'AppBuilder' : 'AwsExplorerLambdaNode'
// Show all log streams without having to choose
const logStreamFilterData: LogStreamFilterResponse = { type: 'all' }
await tailLogGroup(
liveTailRegistry,
source,
liveTailCodeLensProvider,
logGroupInfo,
logStreamFilterData
)
} catch (err) {
if (isError(err as Error, 'ResourceNotFoundException', "LogGroup doesn't exist.")) {
// If we caught this error, then we know `functionConfiguration` actually has a value
throw ToolkitError.chain(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will we get this issue when we deploy the function for the first time, and hit taillog before invoking it?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For I feel this should be a valid use case

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, and that's how the Live Tail API works. It needs an existing log group. We could eventually do something like try once and if it fails we retry (but we might keep retrying forever), but in practice it's better to surface the error, so customers understand it better and retry when needed. Live Tail use case is to "see and filter logs", so it's less likely that people want to use it a lot for a brand new function that doesn't have any invokes yet.

err,
`Unable to fetch logs. Log group for function '${functionConfiguration!.FunctionName}' does not exist. ` +
'Invoking your function at least once will create the log group.'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice hint! Very helpful to surface hints like this.

)
} else {
throw err
}
}
})
)
}
43 changes: 43 additions & 0 deletions packages/core/src/test/awsService/cloudWatchLogs/commands/tailLogGroup.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ import { getTestWindow } from '../../../shared/vscode/window'
import { CloudWatchLogsSettings, uriToKey } from '../../../../awsService/cloudWatchLogs/cloudWatchLogsUtils'
import { DefaultAwsContext, ToolkitError, waitUntil } from '../../../../shared'
import { LiveTailCodeLensProvider } from '../../../../awsService/cloudWatchLogs/document/liveTailCodeLensProvider'
import { PrompterTester } from '../../../shared/wizards/prompterTester'

describe('TailLogGroup', function () {
const testLogGroup = 'test-log-group'
Expand Down Expand Up @@ -142,6 +143,48 @@ describe('TailLogGroup', function () {
assert.strictEqual(stopLiveTailSessionSpy.calledOnce, true)
})

it(`doesn't ask for stream filter if passed as parameter`, async function () {
sandbox.stub(DefaultAwsContext.prototype, 'getCredentialAccountId').returns(testAwsAccountId)
sandbox.stub(DefaultAwsContext.prototype, 'getCredentials').returns(Promise.resolve(testAwsCredentials))

// Returns one frame
async function* generator(): AsyncIterable<StartLiveTailResponseStream> {
yield getSessionUpdateFrame(false, `${testMessage}-1`, 1732276800000)
}

startLiveTailSessionSpy = sandbox
.stub(LiveTailSession.prototype, 'startLiveTailSession')
.returns(Promise.resolve(generator()))

const prompterTester = PrompterTester.init()
.handleInputBox('Provide log event filter pattern', (inputBox) => {
inputBox.acceptValue('filter')
})
.build()

// Set maxLines to 1.
cloudwatchSettingsSpy = sandbox.stub(CloudWatchLogsSettings.prototype, 'get').returns(1)

// The mock stream doesn't 'close', causing tailLogGroup to not return. If we `await`, it will never resolve.
// Run it in the background and use waitUntil to poll its state.
void tailLogGroup(
registry,
testSource,
codeLensProvider,
{
groupName: testLogGroup,
regionName: testRegion,
},
{ type: 'all' }
)
await waitUntil(async () => registry.size !== 0, { interval: 100, timeout: 1000 })

prompterTester.assertCall('Provide log event filter pattern', 1)

assert.strictEqual(startLiveTailSessionSpy.calledOnce, true)
assert.strictEqual(registry.size, 1)
})

it('throws if crendentials are undefined', async function () {
sandbox.stub(DefaultAwsContext.prototype, 'getCredentials').returns(Promise.resolve(undefined))
wizardSpy = sandbox.stub(TailLogGroupWizard.prototype, 'run').callsFake(async function () {
Expand Down
15 changes: 15 additions & 0 deletions packages/core/src/test/awsService/cloudWatchLogs/wizard/tailLogGroupWizard.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,21 @@ describe('TailLogGroupWizard', async function () {
tester.filterPattern.assertShowSecond()
})

it('skips logStream filter when logStream info is provided', async function () {
sandbox.stub(DefaultAwsContext.prototype, 'getCredentialAccountId').returns(testAwsAccountId)
const wizard = new TailLogGroupWizard(
{
groupName: testLogGroupName,
regionName: testRegion,
},
{ type: 'specific', filter: 'log-group-name' }
)
const tester = await createWizardTester(wizard)
tester.regionLogGroupSubmenuResponse.assertDoesNotShow()
tester.logStreamFilter.assertDoesNotShow()
tester.filterPattern.assertShowFirst()
})

it('builds LogGroup Arn properly', async function () {
sandbox.stub(DefaultAwsContext.prototype, 'getCredentialAccountId').returns(testAwsAccountId)
const arn = buildLogGroupArn(testLogGroupName, testRegion)
Expand Down
4 changes: 4 additions & 0 deletions packages/toolkit/.changes/next-release/Feature-48c399ea-d42f-4ea1-8f32-d8bc1486a107.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{
"type": "Feature",
"description": "Support starting CloudWatch Logs Live Tail sessions for a specific Lambda function's log group, without having to find which log group manually, but just starting from the function directly."
valerena marked this conversation as resolved.
Show resolved Hide resolved
}
21 changes: 21 additions & 0 deletions packages/toolkit/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -907,6 +907,10 @@
"command": "aws.appBuilderForFileExplorer.refresh",
"when": "false"
},
{
"command": "aws.appBuilder.tailLogs",
"when": "false"
},
{
"command": "aws.appBuilder.viewDocs",
"when": "false"
Expand Down Expand Up @@ -1670,6 +1674,11 @@
"when": "view =~ /^(aws.explorer|aws.appBuilder|aws.appBuilderForFileExplorer)$/ && viewItem =~ /^(awsRegionFunctionNode|awsRegionFunctionNodeDownloadable|awsCloudFormationFunctionNode)$/",
"group": "0@3"
},
{
"command": "aws.appBuilder.tailLogs",
"when": "view =~ /^(aws.explorer|aws.appBuilder|aws.appBuilderForFileExplorer)$/ && viewItem =~ /^(awsRegionFunctionNode|awsRegionFunctionNodeDownloadable|awsCloudFormationFunctionNode)$/",
"group": "0@4"
},
{
"command": "aws.deleteCloudFormation",
"when": "view == aws.explorer && viewItem == awsCloudFormationNode",
Expand Down Expand Up @@ -2169,6 +2178,11 @@
"command": "aws.appBuilder.searchLogs",
"when": "view =~ /^(aws.explorer|aws.appBuilder|aws.appBuilderForFileExplorer)$/ && viewItem =~ /^(awsRegionFunctionNode|awsRegionFunctionNodeDownloadable|awsCloudFormationFunctionNode)$/",
"group": "inline@2"
},
{
"command": "aws.appBuilder.tailLogs",
"when": "view =~ /^(aws.explorer|aws.appBuilder|aws.appBuilderForFileExplorer)$/ && viewItem =~ /^(awsRegionFunctionNode|awsRegionFunctionNodeDownloadable|awsCloudFormationFunctionNode)$/",
"group": "inline@3"
}
],
"aws.toolkit.auth": [
Expand Down Expand Up @@ -3778,6 +3792,13 @@
"category": "%AWS.title%",
"icon": "$(search-view-icon)"
},
{
"command": "aws.appBuilder.tailLogs",
"title": "%AWS.command.appBuilder.tailLogs%",
"enablement": "isCloud9 || !aws.isWebExtHost",
"category": "%AWS.title%",
"icon": "$(search-show-context)"
},
{
"command": "aws.appBuilder.deploy",
"title": "%AWS.command.appBuilder.deploy%",
Expand Down
Loading