代码之家  ›  专栏  ›  技术社区  ›  garajo

如何延迟流读取调用

  •  10
  • garajo  · 技术社区  · 6 年前

    我还在努力摸索 streams 一般来说。我已经能够流一个大文件使用 multiparty form.on('part') PassThrough , through . through2

    import multiparty from 'multiparty'
    import {
      PassThrough
    } from 'stream';
    import through from 'through'
    import through2 from 'through2'
    
    export function promisedMultiparty(req) {
      return new Promise((resolve, reject) => {
    
        const form = new multiparty.Form()
        const form_files = []
        let q_str = ''
    
        form.on('field', (fieldname, value) => {
          if (value) q_str = appendQStr(fieldname, value, q_str)
        })
    
        form.on('part', async (part) => {
          if (part.filename) {
    
            const pass1 = new PassThrough() // this hangs at 10% 
    
            const pass2 = through(function write(data) { // this hangs from the beginning
                this.queue(data)
              },
              function end() {
                this.queue(null)
              })
    
            const pass3 = through2() // this hangs at 10%
    
            /* 
                // This way works for large files, but I want to defer 
                // invocation
    
                const form_data = new FormData()
                form_data.append(savepath, part, {
                  filename,
                })
    
                const r = request.post(url, {
                  headers: {
                    'transfer-encoding': 'chunked'
                  }
                }, responseCallback(resolve))
                r._form = form
    
            */
    
            form_files.push({
              part: part.pipe(pass1),
              // part: part.pipe(pass2),
              // part: part.pipe(pass3),
            })
    
          } else {
            part.resume()
          }
        })
    
        form.on('close', () => {
          resolve({
            fields: qs.parse(q_str),
            forms: form_files,
          })
        })
    
        form.parse(req)
      })
    }
    

    p、 当然,如果有人能用恰当的术语,标题会更好。谢谢。

    1 回复  |  直到 6 年前
        1
  •  1
  •   Gabriel C. Troia    6 年前

    我相信这是因为你没有使用 through2 正确-也就是说,一旦缓冲区满了就不会清空缓冲区(这就是为什么它在较大的文件上挂起10%,但在较小的文件上工作)。

    我认为这样的实现应该做到:

    const pass2 = through2(function(chunk, encoding, next) {
    
       // do something with the data
    
    
       // Use this only if you want to send the data further to another stream reader 
       // Note - From your implementation you don't seem to need it
       // this.push(data)
    
       // This is what tells through2 it's ready to empty the 
       //  buffer and read more data
       next();
    })