如何将一列或一系列向量转换成稀疏矩阵?

给定一系列SparseVector对象

seq_of_vectors: Array[org.apache.spark.mllib.linalg.SparseVector] = 
    Array(..., (262144,[136034,155107,166596],[0.8164965809277259,0.40824829046386296,0.40824829046386296]), ...

val coo = (seq_of_vectors.map(_.numNonzeros).zipWithIndex.flatMap{case (cnt, idx) => Array.fill(cnt)(idx) },
    seq_of_vectors.map(_.indices).flatten,
    seq_of_vectors.map(_.values).flatten
).zipped.toArray

coo: Array[(Int, Int, Double)] = 
    Array( ..., (28,136034,0.8164965809277259), (28,155107,0.40824829046386296), (28,166596,0.40824829046386296), ...

然后我们使用 fromCOO SparseMatrix . 行数是传递的向量数;列数是最长SparseVector的长度:

SparseMatrix.fromCOO(seq_of_vectors.length,
    seq_of_vectors.map(_.size).max,
    coo)

res223: org.apache.spark.mllib.linalg.SparseMatrix = 
84 x 262144 CSCMatrix
...
(28,136034) 0.8164965809277259
...
(28,155107) 0.40824829046386296
...
(28,166596) 0.40824829046386296
...