我试图从tsv文件中读取一些数据,以便与
Hyperas
,但不管我怎么做,我似乎也会犯同样的错误:
Traceback (most recent call last):
File "/path/to/cnn_search.py", line 233, in <module>
trials=trials)
File "~/miniconda3/lib/python3.6/site-packages/hyperas/optim.py", line 67, in minimize
verbose=verbose)
File "~/miniconda3/lib/python3.6/site-packages/hyperas/optim.py", line 133, in base_minimizer
return_argmin=True),
File "~/miniconda3/lib/python3.6/site-packages/hyperopt/fmin.py", line 312, in fmin
return_argmin=return_argmin,
File "~/miniconda3/lib/python3.6/site-packages/hyperopt/base.py", line 635, in fmin
return_argmin=return_argmin)
File "~/miniconda3/lib/python3.6/site-packages/hyperopt/fmin.py", line 325, in fmin
rval.exhaust()
File "~/miniconda3/lib/python3.6/site-packages/hyperopt/fmin.py", line 204, in exhaust
self.run(self.max_evals - n_done, block_until_done=self.async)
File "~/miniconda3/lib/python3.6/site-packages/hyperopt/fmin.py", line 178, in run
self.serial_evaluate()
File "~/miniconda3/lib/python3.6/site-packages/hyperopt/fmin.py", line 97, in serial_evaluate
result = self.domain.evaluate(spec, ctrl)
File "~/miniconda3/lib/python3.6/site-packages/hyperopt/base.py", line 840, in evaluate
rval = self.fn(pyll_rval)
File "~/temp_model.py", line 218, in keras_fmin_fnct
AttributeError: 'list' object has no attribute 'shape'
从我看到的其他问题来看,这个错误是由于在应该使用numpy数组的地方使用常规数组造成的。所以,我尝试在每一步都将读取的tsv转换为numpy数组:
from hyperas import optim
...
import numpy as np
import csv
def data():
dataPath="/path/to/fm.labeled.10m.txt"
X = []
Y = []
with open(dataPath) as dP:
reader = csv.reader(dP, delimiter="\t")
for row in reader:
#skip the first two columns, and the last column is labels
X.append(np.array(row[2:-1]))
#labels
Y.append(row[-1])
encoder = LabelBinarizer()
Y_categorical = encoder.fit_transform(Y)
#split data into test and train
X_train, X_test, Y_train, Y_test = train_test_split(X, Y_categorical, test_size=0.25)
X_train_np = np.array(X_train)
X_test_np = np.array(X_test)
Y_train_np = np.array([np.array(y) for y in Y_train])
Y_test_np = np.array([np.array(y) for y in Y_test])
return X_train_np, Y_train_np, X_test_np, Y_test_np
...
trials = Trials()
best_run, best_model = optim.minimize(model=model_name,
data=data,
algo=tpe.suggest,
max_evals=numRuns,
trials=trials)
我还认为有一种更有效的方法可以做到这一点,而不需要创建这么多中间数组,这将非常好,因为我将读取数百万行的数据。
我做错什么了?
编辑
以下内容:
Hyperopt wiki
描写
Trials
是的。