我创建了一个scala程序,在数据帧的特定列上应用k-means。数据帧名称为
df_items
而列名是
price
.
import org.apache.spark._
import org.apache.spark.sql.types._
import org.apache.spark.ml.clustering._
import org.apache.spark.ml.feature.VectorAssembler
val df_items = spark.read.format("csv").option("header","true").load(path.csv)
// need to cast because df_items("price") is String
df_items.createGlobalTempView("items")
val price = spark.sql("SELECT cast(price as double) price FROM global_temp.items")
case class Rows(price:Double)
val rows = price.as[Rows]
val assembler = new VectorAssembler().setInputCols(Array("price")).setOutputCol("features")
val data = assembler.transform(rows)
val kmeans = new KMeans().setK(6)
val model = kmeans.fit(data)
val predictions = model.summary.predictions
预测结果:
+------+--------+----------+
| price|features|prediction|
+------+--------+----------+
| 58.9| [58.9]| 0|
| 239.9| [239.9]| 3|
| 199.0| [199.0]| 5|
| 12.99| [12.99]| 0|
| 199.9| [199.9]| 5|
| 21.9| [21.9]| 0|
| 19.9| [19.9]| 0|
| 810.0| [810.0]| 1|
|145.95|[145.95]| 5|
| ... | ... | ... |
我的目标是获得集群(或所有集群)的最小值和最大值。有可能吗?
谢谢,非常感谢