代码之家 › 专栏 › 技术社区 › feeeper

k-hot对多列中的值进行编码

numpy pandas python

feeeper · 技术社区 · 6 年前

我有 pandas.DataFrame :

|   | col_1 | col_2 | col_3 | col_4 |
|:--|:------|:------|:------|:------|
| 0 |   1   |   2   |  NaN  |  NaN  |
| 1 |   3   |   4   |   5   |   6   |
| 2 |   2   |   6   |  NaN  |  NaN  |

我必须将值(1、2、3、4、5、6)转换为列,如果该值在行中,则为行设置1,否则为0:

|   | 1 | 2 | 3 | 4 | 5 | 6 |
|:--|:--|:--|:--|:--|:--|:--|
| 0 | 1 | 1 | 0 | 0 | 0 | 0 |
| 1 | 0 | 0 | 1 | 1 | 1 | 1 |
| 2 | 0 | 1 | 0 | 0 | 0 | 1 |

pd.get_dummies 这里不工作。正如我所见 皮特假人 无法通过热编码 全部的 数据帧列中的值

我怎样才能做到?

3 回复 | 直到 6 年前

Divakar 6 年前

一种方法 broadcasting -

In [67]: df
Out[67]: 
     0    1    2    3
0  1.0  2.0  NaN  NaN
1  3.0  4.0  5.0  6.0
2  2.0  6.0  NaN  NaN

In [68]: constant_set = [1, 2, 3, 4, 5, 6]

In [69]: (df.values[:,:,None] == constant_set).any(1).astype(int)
Out[69]: 
array([[1, 1, 0, 0, 0, 0],
       [0, 0, 1, 1, 1, 1],
       [0, 1, 0, 0, 0, 1]])

# Dataframe output
In [73]: out = (df.values[:,:,None] == constant_set).any(1).astype(int)

In [74]: pd.DataFrame(out, columns=constant_set)
Out[74]: 
   1  2  3  4  5  6
0  1  1  0  0  0  0
1  0  0  1  1  1  1
2  0  1  0  0  0  1

再有一个考虑到记忆效率-

idx = np.searchsorted(constant_set, a)
out = np.zeros((len(df),len(constant_set)),dtype=int)
flattend_idx = idx + out.shape[1]*np.arange(len(idx))[:,None]
out.flat[flattend_idx[idx<len(constant_set)]] = 1

Thijs van Ede 6 年前

您也可以使用 get_dummies 功能如下:

import numpy  as np
import pandas as pd

# The definition of your dataframe
df = pd.DataFrame({'col_1': [1, 3, 2],
                   'col_2': [2, 4, 6],
                   'col_3': [np.NaN, 5, np.NaN],
                   'col_4': [np.NaN, 6, np.NaN]}, dtype=float)

# Get dummies where you leave out the prefix
# This will ensure that all columns of the same value will get the same column name
df = pd.get_dummies(df, columns=['col_1', 'col_2', 'col_3', 'col_4'], prefix='')

# Initialise your result
result = pd.DataFrame()
# Use the groupby method to group on column name
for column, data in df.groupby(level=0, axis=1):
    # Merge data of same columns into one column
    result[column] = data.max(axis=1)

所以我们在这里要做的是在所有的列上应用get-dummies,结果是

   _1.0  _2.0  _3.0  _2.0  _4.0  _6.0  _5.0  _6.0
0     1     1     0     1     0     0     0     0
1     0     0     1     0     1     1     1     1
2     0     1     0     1     0     1     0     1

然后我们合并所有同名的列以获得所需的结果

   _1.0  _2.0  _3.0  _4.0  _5.0  _6.0
0     1     1     0     0     0     0
1     0     0     1     1     1     1
2     0     1     0     0     0     1

user1717828 6 年前

另一种方法使用 pd.melt() :

# Set it up.
import numpy as np; import pandas as pd;
df = pd.DataFrame({'col_1': [1, 3, 2],
                   'col_2': [2, 4, 6],
                   'col_3': [np.NaN, 5, np.NaN],
                   'col_4': [np.NaN, 6, np.NaN]}, dtype=float)

(pd.get_dummies(                     # Pandas' one-hot function
    df.T.melt()                      # Flip DataFrame, then switch from wide to long format.
    .set_index('variable')['value']) # "variable' is the row name (id) in your orig DataFrame.
.groupby('variable')
.sum())                              # Coalesce same ids and add rows together.