Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

~Fix failing tests for invalid control characters in comments~ invalid #617

Closed
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
228 commits
Select commit Hold shift + click to select a range
b4bb91f
test
pelletier Jan 30, 2021
abe1005
wip: string parsing
pelletier Jan 31, 2021
d54ad15
Track ABNF file
pelletier Feb 1, 2021
07aa85e
Refactor to use parser state
pelletier Feb 1, 2021
1c7e9fe
Dotted keys
pelletier Feb 2, 2021
fd96110
Boolean values
pelletier Feb 2, 2021
b96c535
Check for allocs
pelletier Feb 2, 2021
2ab0f8c
Default to use bytes instead of runes
pelletier Feb 2, 2021
7b4d82a
Remove error handling for rune
pelletier Feb 2, 2021
91d7afb
Parse rvalue string
pelletier Feb 2, 2021
bac65cc
Array implementation
pelletier Feb 2, 2021
44f7a7a
Inline tables
pelletier Feb 2, 2021
aae4656
Standard Table
pelletier Feb 2, 2021
7300b6a
Array tables
pelletier Feb 2, 2021
1e8b0dc
Rename to lexer and split in files
pelletier Feb 2, 2021
94ad175
wip
pelletier Feb 2, 2021
b123c35
Add tokens to Document
pelletier Feb 3, 2021
0ee0fe7
Trying the scanner approach
pelletier Feb 4, 2021
ca12c06
wip parsing
pelletier Feb 5, 2021
736a757
Multiline basic string parsing
pelletier Feb 5, 2021
a466f0c
Multiline literal strings
pelletier Feb 6, 2021
540c2a7
Fix parsing bugs + boolean impl
pelletier Feb 6, 2021
165f654
Implement tables
pelletier Feb 6, 2021
b1e11f8
Implement array values
pelletier Feb 6, 2021
9fa2fd4
Implement inline tables
pelletier Feb 6, 2021
89052d6
Very beginning of unmarshaler + builder interface
pelletier Feb 7, 2021
bd8df24
Parse tables
pelletier Feb 7, 2021
a197513
Simple table array
pelletier Feb 8, 2021
70d41bd
Add more tests for unmarshal array tables
pelletier Feb 8, 2021
0e8fd64
Move tests out of the package
pelletier Feb 8, 2021
3488a91
Array values
pelletier Feb 9, 2021
2790964
Bool value
pelletier Feb 9, 2021
7dbf755
Nested arrays
pelletier Feb 10, 2021
0982fd5
Check unhandled error
pelletier Feb 10, 2021
84282bb
Move parser code
pelletier Feb 10, 2021
2660bb8
Test inline tables
pelletier Feb 10, 2021
f6a13d6
wip numbers
pelletier Feb 10, 2021
721fa81
Support numbers
pelletier Feb 10, 2021
27f0aee
Import localtime
pelletier Feb 10, 2021
0dad1a9
Import Unmarshal tests (not passing)
pelletier Feb 10, 2021
e2a07a3
Handle interface field dereference
pelletier Feb 10, 2021
6e79ce6
Reflect write to embeded structs
pelletier Feb 10, 2021
2341b4d
Comment out json-based comparison
pelletier Feb 10, 2021
9ac08fe
DateTime/LocalDate/LocalTime implementation
pelletier Feb 11, 2021
1f41c55
Handle missing fields in structs
pelletier Feb 13, 2021
4657355
WIP constructing pointers
pelletier Feb 19, 2021
629a247
Handle maps
pelletier Feb 19, 2021
052233e
wip: debugging maps
pelletier Feb 19, 2021
7f9822d
Target set methods now check for types
pelletier Feb 19, 2021
978143c
Ensure that slices have been allocated when entering array
pelletier Feb 19, 2021
4526154
wip
pelletier Feb 19, 2021
d24deeb
wip making reflection tests pass
pelletier Feb 20, 2021
c77f1d8
Skip default tags tests
pelletier Feb 20, 2021
bf051f1
Fixed some tests
pelletier Mar 2, 2021
2cee819
Going back and forth on whether types should always be converted
pelletier Mar 2, 2021
f698c10
Convert should only handle specific types
pelletier Mar 8, 2021
c35bcc5
Convert returns pointer if a pointer is passed
pelletier Mar 8, 2021
90f3b65
Support type aliases
pelletier Mar 9, 2021
87b9d1c
Handle overflows
pelletier Mar 9, 2021
a1c9b66
Allocate slice if needed
pelletier Mar 9, 2021
93a74fc
todo: inline tables
pelletier Mar 9, 2021
21d3e85
Playing with an AST
pelletier Mar 13, 2021
d8be04d
Handle simple string slice
pelletier Mar 13, 2021
1fafb71
LF
pelletier Mar 13, 2021
a0548e7
Unmarshal slices of strings
pelletier Mar 14, 2021
fbf01f7
Handle Table
pelletier Mar 14, 2021
fa7ee64
Inline tables
pelletier Mar 14, 2021
3760527
Unmarshal tests
pelletier Mar 14, 2021
04925e4
Handle bools
pelletier Mar 14, 2021
de035f0
Standard tables in parser
pelletier Mar 14, 2021
590d7fa
Parser emits AST node for all kinds of strings
pelletier Mar 14, 2021
9a1cfcd
Replace parser's int or float code with scanner
pelletier Mar 14, 2021
590d674
Unmarshal ints and floats
pelletier Mar 14, 2021
16a336b
Remove todos that don't make sense anymore
pelletier Mar 14, 2021
b8df31d
Comment out date/time tests for now
pelletier Mar 14, 2021
00b2f77
Replace branch with AST version
pelletier Mar 15, 2021
ad64e5d
Update README for v2 work
pelletier Mar 15, 2021
1718142
More todos in README
pelletier Mar 15, 2021
37d06da
Unmarshal into maps
pelletier Mar 15, 2021
844c909
Add todo
pelletier Mar 15, 2021
c6892fc
wip array table
pelletier Mar 15, 2021
f9f9ccb
Basic array table implementation
pelletier Mar 16, 2021
939f889
wip: figuring out unmarshaling to interfaces
pelletier Mar 17, 2021
cb678e6
Passing unmarshal of array table into interfaces
pelletier Mar 18, 2021
a577df2
wip
pelletier Mar 18, 2021
548b128
Fix multiple sub-table in array table
pelletier Mar 18, 2021
fad86a5
Test for sub-table in array table into structs
pelletier Mar 18, 2021
8957a76
Next stop: pointers
pelletier Mar 18, 2021
3e8b8db
Unmarshal into pointers
pelletier Mar 19, 2021
93a7b0d
Skip AST branches that don't exist in the target
pelletier Mar 19, 2021
9ec4e86
Handle struct field name variations
pelletier Mar 19, 2021
ebffe6d
Naive implementation of anonymous structs
pelletier Mar 19, 2021
8b34e54
Improve DOT representation for AST
pelletier Mar 19, 2021
fcc91f2
Progress on date/times
pelletier Mar 22, 2021
ac2d6e2
Handle unmarshalling value to nil ptr.
pelletier Mar 23, 2021
e5d63aa
Add some type conversions
pelletier Mar 23, 2021
b8da9d1
Fix datetime error checking
pelletier Mar 23, 2021
e78ccff
Fix parsing integer 0
pelletier Mar 23, 2021
c6f117c
Handle pointers in slices
pelletier Mar 23, 2021
5b92184
Cast map key type
pelletier Mar 23, 2021
4038ec3
Overflow checks
pelletier Mar 23, 2021
d458ddf
Add TODO
pelletier Mar 23, 2021
0703eeb
Fix bug parsing anonymous structs
pelletier Mar 23, 2021
bfeb32c
Make unmarshal to interface{} consistent with encoding/json
pelletier Mar 24, 2021
a3b7e1e
Fix table array into pointer to slice
pelletier Mar 24, 2021
a25f636
Add array support todo
pelletier Mar 24, 2021
a0d031a
Arrays support
pelletier Mar 25, 2021
dd58376
Support TextUnmarshaler
pelletier Mar 25, 2021
43fc2fa
Factor pointer handling
pelletier Mar 25, 2021
ad538d9
Delete reflectbuild
pelletier Mar 25, 2021
8a8d123
First benchmark!
pelletier Mar 25, 2021
1bae751
Linear array storage for AST
pelletier Mar 25, 2021
17299c9
Update readme
pelletier Mar 25, 2021
1da2fc7
Minimal shared cache for struct field paths
pelletier Mar 26, 2021
9d3a912
Remove unused interface
pelletier Mar 26, 2021
1d332cd
Add documentation for the AST
pelletier Mar 26, 2021
0fcf06e
Update todo
pelletier Mar 26, 2021
4efec6b
Add github actions workflow
pelletier Mar 26, 2021
ffc7d3b
Port codeql
pelletier Mar 26, 2021
6c8adbc
Remove azure pipeline
pelletier Mar 26, 2021
e75f231
Add v2-wip to codeql branches
pelletier Mar 26, 2021
f4ac7f7
Multi OS testing
pelletier Mar 26, 2021
47611ff
Shorter names for CI
pelletier Mar 26, 2021
3f23ab9
Revert os/go version to fit in github pop-up
pelletier Mar 26, 2021
390927a
Reuse AST storage between top-level expressions
pelletier Mar 26, 2021
636a75f
Import tomltestgen
pelletier Mar 26, 2021
317b36b
Add back license
pelletier Mar 26, 2021
e5a091a
Don't depend on my computer path
pelletier Mar 28, 2021
72c999e
Fix trailing commas in arrays
pelletier Mar 28, 2021
9a436c7
Remove logging in test
pelletier Mar 28, 2021
7dc5550
Fix multiline basic string parsing
pelletier Mar 28, 2021
b24eb93
Fix literal multiline parsing
pelletier Mar 28, 2021
829c005
Fix unicode decoding
pelletier Mar 28, 2021
da21b0a
wip: correctness pass on the AST
pelletier Mar 29, 2021
2ddbf6b
Implement duplicate and key types check
pelletier Mar 29, 2021
6165b94
Identify test helper functions (#487)
moorereason Mar 30, 2021
7d8ea80
Fix scanning of float with leading zero (#486)
moorereason Mar 30, 2021
269b742
Enable race condition detector in CI
pelletier Mar 30, 2021
7f016ef
Test for #484
pelletier Mar 30, 2021
c3fc668
Test for #458
pelletier Mar 30, 2021
78389c6
Test for #475
pelletier Mar 30, 2021
51d78a5
Fix unmarshaling of literal keys
pelletier Mar 30, 2021
2714786
Add decoder interface
pelletier Mar 30, 2021
72a1afd
Add some unsafe helper to track errors
pelletier Mar 30, 2021
cf288a5
Wip errors reporting
pelletier Mar 30, 2021
e5255a5
Set bytes in ReferenceFile benchmark to show throughput results (#489)
moorereason Mar 30, 2021
bcd5333
Enable ci on v2-wip branch pull requests
pelletier Mar 30, 2021
18d45c4
wip: decoder errors
pelletier Mar 30, 2021
32da85a
Decoding error position tracking
pelletier Mar 31, 2021
739ceda
Move github pull request template
pelletier Mar 31, 2021
7ccacf1
Test for #252
pelletier Mar 31, 2021
5d90598
CI: add dependabot
pelletier Mar 31, 2021
4a4c2c2
Update readme
pelletier Mar 31, 2021
92b16ca
Simplify context implementation and fix new lines bug
pelletier Mar 31, 2021
5f877c5
Add additional dataset to benchmarks (#490)
moorereason Apr 1, 2021
af00765
Address golangci-lint warnings in unmarshal_imported_test.go (#493)
jidicula Apr 5, 2021
18af62d
Golangci-lint v2 part one (#492)
vincentserpoul Apr 7, 2021
275e366
decoder: handle casting local date into time.Time
pelletier Apr 8, 2021
3771400
V2 Marshaler MVP (#495)
pelletier Apr 8, 2021
f237898
encoder: added test for #287
pelletier Apr 8, 2021
ca41df4
encoder: only create empty map when target exists
pelletier Apr 8, 2021
84f9e9b
ci: run benchmark tests
pelletier Apr 8, 2021
e1f0354
encoder: simplify quoted strings escaping
pelletier Apr 9, 2021
466bfe8
encoder: inline tables create map in nil interface (#496)
moorereason Apr 9, 2021
ed1f9ed
Add sanity check tests to benchmark dataset (#497)
moorereason Apr 9, 2021
9e122af
encoder: support multiline strings + local options
pelletier Apr 10, 2021
59cddbc
Golangci-lint v2 part two (#498)
vincentserpoul Apr 15, 2021
2eff2d0
Rename branch v2-wip -> v2
pelletier Apr 15, 2021
0537b92
decoder: add test for #507
pelletier Apr 15, 2021
080baa8
golangci-lint: localtime (#509)
vincentserpoul Apr 15, 2021
b86b890
decoder: handle private anonymous structs
pelletier Apr 15, 2021
9bc4641
ci-lint: disable ifshort
pelletier Apr 15, 2021
24b62eb
Simplify scanFollows usage (#510)
moorereason Apr 15, 2021
a7b50eb
Tidy (#511)
moorereason Apr 15, 2021
a713a96
Add more newline tests for scanner (#515)
moorereason Apr 16, 2021
dca2103
golangci-lint: marshaler (#516)
vincentserpoul Apr 21, 2021
9b67e40
decoder: strict mode (#512)
pelletier Apr 21, 2021
ee102a3
decoder: fix time fractional parsing
pelletier Apr 21, 2021
32c1a8d
encoder: move nspow into the parseLocalTime
pelletier Apr 21, 2021
6fe332a
Encoder inline tables (#519)
pelletier Apr 21, 2021
9ba5299
Encoder multiline array (#520)
pelletier Apr 22, 2021
21445f5
Add test for issue #424
pelletier Apr 22, 2021
2b1c52d
golangci-lint: decoder/unmarshal (#518)
vincentserpoul Apr 22, 2021
e443b4f
encoder: support TextMarshaler (#522)
pelletier Apr 22, 2021
466faaa
golangci-lint: marshaler, strict (#523)
vincentserpoul Apr 23, 2021
a533331
v2: benchdiff (#524)
pelletier Apr 23, 2021
931f02a
encoder: support indentation (#525)
pelletier Apr 23, 2021
1e80267
parser: require \n after parsing integer in kv (#527)
pelletier Apr 24, 2021
201d5dd
golangci-lint: misc (#529)
vincentserpoul Apr 28, 2021
3f2bb0b
golangci-lint (#530)
vincentserpoul May 7, 2021
4545a3e
ci: remove benchmarks
pelletier May 8, 2021
ea225df
v2: errors (#534)
pelletier May 8, 2021
45ea200
Readme (#535)
pelletier May 8, 2021
3db329a
ci: basic github action for coverage (#537)
pelletier May 9, 2021
95c701b
Increase test coverage (#538)
pelletier May 11, 2021
d276c42
Run coverage test on branches only
pelletier May 11, 2021
67852cf
Clarify default struct tag being unsupported (#543)
pelletier May 15, 2021
238a6fe
Add links to proper feedback channels
pelletier May 15, 2021
c2d1fd8
Fix timezone detection when time has fractional component (#544)
pelletier May 21, 2021
840df4a
seen tracker: use array for storage (#545)
pelletier May 26, 2021
11f022a
Fix parser skipping over the whole file (#547)
pelletier May 30, 2021
250e073
Stack-based unmarshaler (#546)
pelletier May 31, 2021
b202375
Add benchmarks results to readme (#548)
pelletier Jun 1, 2021
b0d6c62
Don't use bytes.Buffer when not necessary (#549)
pelletier Jun 1, 2021
f3bb20e
Benchmark marshal (#550)
pelletier Jun 2, 2021
618f018
AST Tweaks (#551)
pelletier Jun 4, 2021
773f101
Unmarshal recursive structs (#557)
pelletier Jun 8, 2021
f6b38c3
Provide own implementation of Local* (#558)
pelletier Jun 9, 2021
9c24fbe
Set up Dependabot for GitHub actions and docker (#570)
mmorel-35 Jul 20, 2021
a93b34d
Unicode parsing optimization (#568)
kkHAIKE Jul 21, 2021
8be357d
Add LocalTime to interface{} decode support (#567)
kkHAIKE Jul 21, 2021
fa07960
Add installation instructions (#572)
pelletier Jul 27, 2021
69ab7e1
Go 1.17 release (#574)
pelletier Aug 17, 2021
1230ca4
unmarshal: make copy of non addressable values (#576)
pelletier Sep 1, 2021
40cfb6f
parser: don't crash on unterminated table key (#580)
pelletier Sep 6, 2021
7e2fa1b
unmarshal: fix non-terminated array error
pelletier Sep 7, 2021
4a5ae9e
errors: fix context generation with only one line
pelletier Sep 7, 2021
a0d685d
unmarshal: don't crash on unterminated inline table (#587)
pelletier Sep 8, 2021
f34c9c3
scanner: fix error reporting for last comments (#591)
pelletier Sep 9, 2021
fa56f48
parser: don't overflow when parsing bad times (#593)
pelletier Sep 9, 2021
ee9b902
unmarshal: convert ints if target type is compatible (#594)
pelletier Sep 10, 2021
476492a
unmarshal: support lowercase 'T' and 'Z' in date-time parsing (#601)
moorereason Sep 25, 2021
62acca2
tomltestgen: add toml-test unit test generation command (#610)
moorereason Oct 4, 2021
91027d8
feat(scanComment): Add error return value
jidicula Oct 5, 2021
f7f02b0
fix(comment test): Fail if DEL 0x7f char is found
jidicula Oct 5, 2021
851d9c4
fix(comment test): Fail if LF 0x0a char is found
jidicula Oct 5, 2021
6bc6702
fix(comment test): Fail if NUL 0x00 char is found
jidicula Oct 5, 2021
1c407d1
fix(comment test): Fail if US 0x1f char is found
jidicula Oct 5, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
* text=auto

benchmark/benchmark.toml text eol=lf
19 changes: 19 additions & 0 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
<!--

Thank you for your pull request!

Please read the Code changes section of the CONTRIBUTING.md file,
and make sure you have followed the instructions.

https://github.com/pelletier/go-toml/blob/v2/CONTRIBUTING.md#code-changes

-->

Explanation of what this pull request does.

More detailed description of the decisions being made and the reasons why (if
the patch is non-trivial).

---

Paste `benchstat` results here
17 changes: 17 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
version: 2
updates:
- package-ecosystem: gomod
directory: /
schedule:
interval: daily
open-pull-requests-limit: 10
- package-ecosystem: github-actions
directory: /
schedule:
interval: daily
open-pull-requests-limit: 10
- package-ecosystem: docker
directory: /
schedule:
interval: daily
open-pull-requests-limit: 10
67 changes: 67 additions & 0 deletions .github/workflows/codeql-analysis.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
# For most projects, this workflow file will not need changing; you simply need
# to commit it to your repository.
#
# You may wish to alter this file to override the set of languages analyzed,
# or to provide custom queries or build logic.
#
# ******** NOTE ********
# We have attempted to detect the languages in your repository. Please check
# the `language` matrix defined below to confirm you have the correct set of
# supported CodeQL languages.
#
name: "CodeQL"

on:
push:
branches: [ master, v2 ]
pull_request:
# The branches below must be a subset of the branches above
branches: [ master ]
schedule:
- cron: '26 19 * * 0'

jobs:
analyze:
name: Analyze
runs-on: ubuntu-latest

strategy:
fail-fast: false
matrix:
language: [ 'go' ]
# CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python' ]
# Learn more:
# https://docs.github.com/en/free-pro-team@latest/github/finding-security-vulnerabilities-and-errors-in-your-code/configuring-code-scanning#changing-the-languages-that-are-analyzed

steps:
- name: Checkout repository
uses: actions/checkout@v2

# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v1
with:
languages: ${{ matrix.language }}
# If you wish to specify custom queries, you can do so here or in a config file.
# By default, queries listed here will override any specified in a config file.
# Prefix the list here with "+" to use these queries and those in the config file.
# queries: ./path/to/local/query, your-org/your-repo/queries@main

# Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
# If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild
uses: github/codeql-action/autobuild@v1

# ℹ️ Command-line programs to run using the OS shell.
# 📚 https://git.io/JvXDl

# ✏️ If the Autobuild fails above, remove it and uncomment the following three lines
# and modify them (or add more) to build your code if your project
# uses a compiled language

#- run: |
# make bootstrap
# make release

- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v1
20 changes: 20 additions & 0 deletions .github/workflows/coverage.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
name: coverage
on:
pull_request:
branches:
- v2

jobs:
report:
runs-on: "ubuntu-latest"
name: report
steps:
- uses: actions/checkout@master
with:
fetch-depth: 0
- name: Setup go
uses: actions/setup-go@master
with:
go-version: 1.16
- name: Run tests with coverage
run: ./ci.sh coverage -d "${GITHUB_BASE_REF-HEAD}"
25 changes: 25 additions & 0 deletions .github/workflows/workflow.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
name: test
on:
push:
branches:
- v2
pull_request:
branches:
- v2

jobs:
build:
strategy:
matrix:
os: [ 'ubuntu-latest', 'windows-latest', 'macos-latest']
go: [ '1.16', '1.17' ]
runs-on: ${{ matrix.os }}
name: ${{ matrix.go }}/${{ matrix.os }}
steps:
- uses: actions/checkout@master
- name: Setup go ${{ matrix.go }}
uses: actions/setup-go@master
with:
go-version: ${{ matrix.go }}
- name: Run unit tests
run: go test -race ./...
84 changes: 84 additions & 0 deletions .golangci.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
[service]
golangci-lint-version = "1.39.0"

[linters-settings.wsl]
allow-assign-and-anything = true

[linters-settings.exhaustive]
default-signifies-exhaustive = true

[linters]
disable-all = true
enable = [
"asciicheck",
"bodyclose",
"cyclop",
"deadcode",
"depguard",
"dogsled",
"dupl",
"durationcheck",
"errcheck",
"errorlint",
"exhaustive",
# "exhaustivestruct",
"exportloopref",
"forbidigo",
# "forcetypeassert",
"funlen",
"gci",
# "gochecknoglobals",
"gochecknoinits",
"gocognit",
"goconst",
"gocritic",
"gocyclo",
"godot",
"godox",
# "goerr113",
"gofmt",
"gofumpt",
"goheader",
"goimports",
"golint",
"gomnd",
# "gomoddirectives",
"gomodguard",
"goprintffuncname",
"gosec",
"gosimple",
"govet",
# "ifshort",
"importas",
"ineffassign",
"lll",
"makezero",
"misspell",
"nakedret",
"nestif",
"nilerr",
# "nlreturn",
"noctx",
"nolintlint",
#"paralleltest",
"prealloc",
"predeclared",
"revive",
"rowserrcheck",
"sqlclosecheck",
"staticcheck",
"structcheck",
"stylecheck",
# "testpackage",
"thelper",
"tparallel",
"typecheck",
"unconvert",
"unparam",
"unused",
"varcheck",
"wastedassign",
"whitespace",
# "wrapcheck",
# "wsl"
]
Loading