Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cut not producing output equivalent to put+drop #4830

Closed
philrz opened this issue Oct 26, 2023 · 1 comment · Fixed by #4832
Closed

cut not producing output equivalent to put+drop #4830

philrz opened this issue Oct 26, 2023 · 1 comment · Fixed by #4832
Assignees
Labels
bug Something isn't working

Comments

@philrz
Copy link
Contributor

philrz commented Oct 26, 2023

Repro is with Zed commit e03665e.

Start with the successful verification example shown in #4555 (comment) that uses attached test data scan.json.gz.

$ zq -version
Version: v1.10.0-20-ge03665e6

$ zq -Z 'over this | ports:=collect(ports[0]) by ip | put this[ip]:=ports | drop ip,ports' scan.json.gz 
{
    "192.168.5.53": [
        {
            port: 515,
            proto: "tcp",
            status: "open",
            reason: "syn-ack",
            ttl: 64
        },
        {
            port: 21,
            proto: "tcp",
            status: "open",
            reason: "syn-ack",
            ttl: 64
        }
    ]
}
{
    "192.168.5.49": [
        {
            port: 49152,
            proto: "tcp",
            status: "open",
            reason: "syn-ack",
            ttl: 64
        },
        {
            port: 3401,
            proto: "tcp",
            status: "open",
            reason: "syn-ack",
            ttl: 64
        }
    ]
}

In #4795 (comment), @mattnibs pointed out that the put/drop combo could be improved by replacing it with cut as shown below. However, this produces incorrect output: Notice that instead of the separate keys "192.168.5.49" and "192.168.5.53", the same key "192.168.5.53" appears twice.

$ zq -Z 'over this | ports:=collect(ports[0]) by ip | cut this[ip]:=ports' scan.json.gz 
{
    "192.168.5.53": [
        {
            port: 515,
            proto: "tcp",
            status: "open",
            reason: "syn-ack",
            ttl: 64
        },
        {
            port: 21,
            proto: "tcp",
            status: "open",
            reason: "syn-ack",
            ttl: 64
        }
    ]
}
{
    "192.168.5.53": [
        {
            port: 49152,
            proto: "tcp",
            status: "open",
            reason: "syn-ack",
            ttl: 64
        },
        {
            port: 3401,
            proto: "tcp",
            status: "open",
            reason: "syn-ack",
            ttl: 64
        }
    ]
}

FWIW, I've noticed that if I run this in a loop sometimes the key "192.168.5.49" will show up in both spots instead.

@philrz philrz added the bug Something isn't working label Oct 26, 2023
@philrz
Copy link
Contributor Author

philrz commented Nov 5, 2023

Verified in Zed commit 07893a2.

The repro example above that uses cut now shows separate keys in the output.

$ zq -version
Version: v1.10.0-36-g07893a2b

$ zq -Z 'over this | ports:=collect(ports[0]) by ip | cut this[ip]:=ports' scan.json.gz
{
    "192.168.5.53": [
        {
            port: 515,
            proto: "tcp",
            status: "open",
            reason: "syn-ack",
            ttl: 64
        },
        {
            port: 21,
            proto: "tcp",
            status: "open",
            reason: "syn-ack",
            ttl: 64
        }
    ]
}
{
    "192.168.5.49": [
        {
            port: 49152,
            proto: "tcp",
            status: "open",
            reason: "syn-ack",
            ttl: 64
        },
        {
            port: 3401,
            proto: "tcp",
            status: "open",
            reason: "syn-ack",
            ttl: 64
        }
    ]
}

Thanks @mattnibs!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants