Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Master pr kz csv changes #6446

Closed
wants to merge 5 commits into from

Conversation

bradfordben
Copy link
Contributor

Standardized (with the lack of standards, https://tools.ietf.org/html/rfc4180) the creating and reading of CSV files. Properly escaping charters.

We needed to be able to do this as the current csv code could not handle escaped , correctly (Would replace them with ;)

This was needed for a kazoo tasks callflow module (separate pr to come)to allow the user to create a callflow with a dynamic flow you needed to be able to take flow as an escaped json string.

Example json data in a csv row would be
not json,"{""data"": { ""id"": ""2ab9ffb3bad31b8bb1d38eaf840f8dcc"", ""timeout"": 20, ""can_call_self"": false, ""delay"": 0, ""strategy"": ""simultaneous"" }, ""module"": ""user"", ""children"": {}}"

@bradfordben
Copy link
Contributor Author

@jamesaimonetti Think the build issue here is unrelated issue ?

Copy link
Member

@jamesaimonetti jamesaimonetti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The CI failure is related. pqc_cb_rates takes a list of 3 rate json objects and tries to create a CSV from them which is now mal-formed:

"_id","direction","prefix","pvt_type","rate_cost","ratedeck_id","routes"
"XX-1222","inbound","1222","rate","1.0","custom",""
"XX-1333","inbound","1333","rate","2.0","custom","^\+?1333.+$"
"XX-1444","inbound","1444","rate","3.0","custom","["^\\+?1444.+$"]"

You'll notice the last row's last column is unescaped quotes. This gets decoded on KAZOO's side to <<"[">> which is...not great...as a route regexp.

core/kazoo_csv/test/kz_csv_tests.erl Outdated Show resolved Hide resolved
@bradfordben bradfordben force-pushed the master-pr_kz_csv_changes branch from 6af02bd to 9e09185 Compare April 16, 2020 00:47
@bradfordben
Copy link
Contributor Author

@jamesaimonetti I have change things to always surround cells in double quotes and fixed the json to/from csv error so it should be good for a second look when you have a chance.

@bradfordben bradfordben force-pushed the master-pr_kz_csv_changes branch from 9e09185 to aad737c Compare April 16, 2020 02:48
@bradfordben bradfordben force-pushed the master-pr_kz_csv_changes branch from aad737c to cd4431a Compare April 27, 2020 21:22
@bradfordben
Copy link
Contributor Author

bradfordben commented Apr 28, 2020

Im not sure why make release failed, It looks like it hit a system error when trying to load a view from couch during account validation. Cant see why the CSV changes would have caused this.

Is this an error pulled in from master?

@bradfordben bradfordben force-pushed the master-pr_kz_csv_changes branch from cd4431a to 1e7b275 Compare April 29, 2020 22:25
@bradfordben
Copy link
Contributor Author

rebasing on master

jamesaimonetti pushed a commit that referenced this pull request Apr 29, 2020
Standardized (with the lack of standards,
https://tools.ietf.org/html/rfc4180) the creating and reading of CSV
files and properly escaping characters.

Prior to this, the CSV code could not handle escaped `,`
correctly (Would replace them with `;`). This is needed for including
JSON in CSV cells.
@jamesaimonetti
Copy link
Member

Merged, thanks!

@bradfordben
Copy link
Contributor Author

@jamesaimonetti Thanks for reviewing. Would you like me to backport this to 4.3 or not?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants