Imbalance dataset detection enhancement

[sepandhaghighi]

Description

PyCM imbalance detection is weak, example :

>>> cm = ConfusionMatrix(matrix={0:{0:60,1:0,2:0,3:0},1:{0:0,1:20,2:0,3:0},2:{0:0,1:0,2:20,3:0},3:{0:0,1:0,2:0,3:20}})
>>> cm.print_matrix()
Predict  0        1        2        3        
Actual
0        60       0        0        0        

1        0        20       0        0        

2        0        0        20       0        

3        0        0        0        20       


>>> cm.imbalance
False

[sepandhaghighi]

There is no standard method for imbalance dataset detection in multi-class mode, but I suggest the following method :

Any idea ?

Note 1 : P : Condition Positive , C : All Classes
Note 2 : 20% comes from 40/60 in binary mode

[sepandhaghighi]

@alirezazolanvari @sadrasabouri
Please join this discussion

[sepandhaghighi]

@ssheikholeslami
I also invite you to this discussion ;-)

[sadrasabouri]

There is no standard method for imbalance dataset detection in multi-class mode, but I suggest the following method :

Any idea ?

Note 1 : P : Condition Positive , C : All Classes
Note 2 : 20% comes from 40/60 in binary mode

How about defining a variable for each class that present it's weight in main data set and calculate the variance for it to determine if the data set is imbalance or not.

[alirezazolanvari]

There is no standard method for imbalance dataset detection in multi-class mode, but I suggest the following method :

Any idea ?

Note 1 : P : Condition Positive , C : All Classes
Note 2 : 20% comes from 40/60 in binary mode

I think only the population of the most and the least populated classes play role in imbalance detection.
What is the added value of calculating the average or the variance of the population of other classes?

[sepandhaghighi]

I think only the population of the most and the least populated classes play role in imbalance detection.
What is the added value of calculating the average or the variance of the population of other classes?

I think the method that you mentioned here is not accurate enough at all cases.

1. Its hard to select a general scaling threshold
2. This method bypass equal distribution definition

[sepandhaghighi]

How about defining a variable for each class that present it's weight in main data set and calculate the variance for it to determine if the data set is imbalance or not.

Weight is a good idea!
We should consider a method to control threshold by this weight vector

[sadrasabouri]

How about defining a variable for each class that present it's weight in main data set and calculate the variance for it to determine if the data set is imbalance or not.

Weight is a good idea!
We should consider a method to control threshold by this weight vector

We can calculate E, weighted and then divide E by sum of the wights then E should be normaled respected to weights.

and about @alirezazolanvari 's idea, i couldn't find any exception (at least at first view) could you please write down an example that doesn't seems Imbalance but has large difference between the min and max and vice versa.

[sepandhaghighi]

and about @alirezazolanvari 's idea, i couldn't find any exception (at least at first view) could you please write down an example that doesn't seems Imbalance but has large difference between the min and max and vice versa.

There is no standard method for imbalance dataset detection and @alirezazolanvari idea is not wrong , but take a look at this example :

Class 1 : 3900
Class 2 : 2700
Class 3 : 2700
Class 4 : 2700

The current method recognizes this distribution as balanced !! (even with ratio of 1.5)

[alirezazolanvari]

We can also define an is_imbalanced flag so that the user can indicate whether the concerned dataset is imbalanced or not. As long as the user does not provide any information in this regard, the automatic detection algorithm will be used.

[sadrasabouri]

If it's OK I can work on this issue for version 3.3.

[sepandhaghighi]

If it's OK I can work on this issue for version 3.3.

🥇