代码之家 › 专栏 › 技术社区 › Maryam Koulaei

mc。windows不支持cores>1

multicore parallel-processing windows r

Maryam Koulaei · 技术社区 · 7 年前

我是R编程新手,我有下面这样的代码,我知道windows不支持多核,但我不知道如何更改这部分代码。

rpl <- unlist( lapply( waydf$geometry$coordinates , nrow ) ) # row per line
 waydf <- waydf[ rpl > 1 , ]
ll <- parallel::mclapply( waydf$geometry$coordinates , st_linestring,
                         mc.cores =parallel::detectCores() - 1  )
outdf <- sf::st_sf(
line_geometry = sf::st_sfc( ll , crs = epsg ) ,
osm_id = waydf$id
)

3 回复 | 直到 7 年前

Aaron - mostly inactive 7 年前

如果你想做的只是使代码不并行运行,你只需要告诉它使用1个内核,然后它就会使用 lapply 在引擎盖下。

ll <- parallel::mclapply(waydf$geometry$coordinates, st_linestring, mc.cores = 1)

mclapply

ll <- lapply(waydf$geometry$coordinates, st_linestring)

CPak 7 年前

你必须澄清什么 st_linestring waydf$geometry$coordinates 但没有指定任何参数,例如 st_linestring(waydf$geometry$coordinates[i])

在Windows中,您将使用 parLapply mclapply .

# original
ll <- parallel::mclapply( waydf$geometry$coordinates , st_linestring,
                         mc.cores =parallel::detectCores() - 1  )

# replace the above with all of the below
library(parallel)
cl <- makeCluster(detectCores())
cl <- clusterEvalQ(cl, { library(sf) })  # you need to export packages as well
# cl <- clusterExport(cl, "st_linestring")  # each worker is a new environment, you will need to export variables/functions to
ll <- parallel::parLapply(cl, waydf$geometry$coordinates, function(i) st_linestring)    # if st_linestring takes arguments then st_linestring(x)
stopCluster(cl)

编辑由于st_linestring是来自包sf的函数,因此导出sf就足够了

第二次编辑

rpl <- unlist( lapply( waydf$geometry$coordinates , nrow ) ) # row per line
 waydf <- waydf[ rpl > 1 , ]

library(parallel)
cl <- makeCluster(detectCores())
cl <- clusterEvalQ(cl, { library(sf) })  # you need to export packages as well
# cl <- clusterExport(cl, "st_linestring")  # each worker is a new environment, you will need to export variables/functions to
ll <- parallel::parLapply(cl, waydf$geometry$coordinates, function(i) st_linestring)    # if st_linestring takes arguments then st_linestring(x)
stopCluster(cl)

outdf <- sf::st_sf(
line_geometry = sf::st_sfc( ll , crs = epsg ) ,
osm_id = waydf$id
)

HenrikB 7 年前

问题是你确实 cl <- ... cl 一次,然后重复使用。

library("parallel")
cl <- makeCluster(detectCores())
clusterEvalQ(cl, { library("sf") })
clusterExport(cl, "st_linestring")
res <- parallel::parLapply(cl, X = waydf$geometry$coordinates, 
                         fun = function(i) st_linestring)
stopCluster(cl)

信息 Error in checkCluster(cl): not a valid cluster 你用你的代码得到的是因为在你这样做之后 cl <- clusterEvalQ(cl, { library("sf") }) cluster 对象