Drain.py modified (Bug Fix) #57

kvlksgjp1 · 2020-04-06T20:09:46Z

When I use 'Drain.py' for my database log(hadoop log) I found there's some bug like this..

Traceback (most recent call last):
  File "./run_method.py", line 55, in <module>
    parser.parse(os.path.basename(setting['log_file']))
  File "../logparser/Drain/Drain.py", line 285, in parse
    self.outputResult(logCluL)
  File "../logparser/Drain/Drain.py", line 216, in outputResult
    self.df_log["ParameterList"] = self.df_log.apply(self.get_parameter_list, axis=1)
  File "/usr/local/lib/python2.7/dist-packages/pandas/core/frame.py", line 4877, in apply
    ignore_failures=ignore_failures)
  File "/usr/local/lib/python2.7/dist-packages/pandas/core/frame.py", line 4990, in _apply_standard
    result = self._constructor(data=results, index=index)
  File "/usr/local/lib/python2.7/dist-packages/pandas/core/frame.py", line 330, in __init__
    mgr = self._init_dict(data, index, columns, dtype=dtype)
  File "/usr/local/lib/python2.7/dist-packages/pandas/core/frame.py", line 461, in _init_dict
    return _arrays_to_mgr(arrays, data_names, index, columns, dtype=dtype)
  File "/usr/local/lib/python2.7/dist-packages/pandas/core/frame.py", line 6173, in _arrays_to_mgr
    return create_block_manager_from_arrays(arrays, arr_names, axes)
  File "/usr/local/lib/python2.7/dist-packages/pandas/core/internals.py", line 4642, in create_block_manager_from_arrays
    construction_error(len(arrays), arrays[0].shape, axes, e)
  File "/usr/local/lib/python2.7/dist-packages/pandas/core/internals.py", line 4604, in construction_error
    raise e
ValueError: could not broadcast input array from shape (3) into shape (4)

I traced the cause of this bug.
And finally I found the way to fix this bug.

in Drain.py - 215 line

if self.keep_para:
            self.df_log["ParameterList"] = self.df_log.apply(self.get_parameter_list, axis=1)

I changed this code like this..

if self.keep_para:
            parameter_list=[]
            for i, row in self.df_log.iterrows():
                parameter_list.append(self.get_parameter_list(row))
            p_series=pd.Series(parameter_list, name='ParameterList')
            self.df_log = pd.concat([self.df_log, p_series], axis=1)

these two codes do exactly same thing.
But my version does't make bug.

* Fix PRs: #86, #85, #83, #80, #78, #65, #57, #53, #52, #51, #49 * Fix issues: #98, #79, #72, #84, #75, #71, #69, #56, #50, #45, #16, #82

* Fix PRs: fix #86, fix #85, #83, #80, #78, #65, #57, #53, #52, #51, #49 * Fix issues: #98, #79, #72, #84, #75, #71, #69, #56, #50, #45, #16, #82

* Fix PRs: merge #86, merge #85, merge #83, merge #80, merge #78, merge #65, merge #57, merge #53, merge #52, merge #51, merge #49 * Fix issues: #98, #79, #72, #84, #75, #71, #69, #56, #50, #45, #16, #82

* Close PRs: close #86, #85, #83, #80, #78, #65, #57, #53, #52, #51, #49 * Fix issues: fix #98, #79, #72, #84, #75, #71, #69, #56, #50, #45, #16, #82

* Close PRs: close #86, #85, #83, #80, #78, #65, #57, #53, #52, #51, #49 * Fix issues: fix #98, #79, #72, #84, #75, #71, #69, #56, #50, #45, #16, #82 * Release pypi package wheel

* Close PRs: close #86, #85, #83, #80, #78, #65, #57, #53, #52, #51, #49 * Fix issues: fix #98, #79, #72, #84, #75, #71, #69, #56, #50, #45, #16, #82 * Release package wheel to pypi

Drain.py modified (Bug Fix)

253a94b

xpai pushed a commit that referenced this pull request Sep 5, 2023

Update logparser v1.0.0 to support Python 3 compatibility

a22c324

* Fix PRs: #86, #85, #83, #80, #78, #65, #57, #53, #52, #51, #49 * Fix issues: #98, #79, #72, #84, #75, #71, #69, #56, #50, #45, #16, #82

xpai pushed a commit that referenced this pull request Sep 5, 2023

Update logparser v1.0.0 to support Python 3 compatibility

641b5d3

* Fix PRs: #86, #85, #83, #80, #78, #65, #57, #53, #52, #51, #49 * Fix issues: #98, #79, #72, #84, #75, #71, #69, #56, #50, #45, #16, #82

xpai pushed a commit that referenced this pull request Sep 5, 2023

Update logparser v1.0.0 to support Python 3 compatibility

be4fa3e

* Fix PRs: fix #86, fix #85, #83, #80, #78, #65, #57, #53, #52, #51, #49 * Fix issues: #98, #79, #72, #84, #75, #71, #69, #56, #50, #45, #16, #82

xpai pushed a commit that referenced this pull request Sep 5, 2023

Update logparser v1.0.0 to support Python 3 compatibility

3bfb46f

* Close PRs: close #86, #85, #83, #80, #78, #65, #57, #53, #52, #51, #49 * Fix issues: fix #98, #79, #72, #84, #75, #71, #69, #56, #50, #45, #16, #82

zhujiem closed this Sep 5, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Drain.py modified (Bug Fix) #57

Drain.py modified (Bug Fix) #57

kvlksgjp1 commented Apr 6, 2020

Drain.py modified (Bug Fix) #57

Drain.py modified (Bug Fix) #57

Conversation

kvlksgjp1 commented Apr 6, 2020