代码之家  ›  专栏  ›  技术社区  ›  Kurt Peek

使用Lodash memoize“Javascript堆内存不足”

  •  3
  • Kurt Peek  · 技术社区  · 6 年前

    longest palindromic subsequence 通过将记忆应用于递归解决方案来使用Javascript的问题。这是递归解, longestPalindromicSubsequence.js :

    function longestPalindromicSubsequence(string, start = 0, end = string.length) {
      if (end < start) { return 0; }
      if (start === end) { return 1; }
      if (string[start] === string[end]) {
        return 2 + longestPalindromicSubsequence(string, start + 1, end - 1);
      }
      return Math.max(
        longestPalindromicSubsequence(string, start + 1, end),
        longestPalindromicSubsequence(string, start, end - 1),
      );
    }
    
    module.exports = longestPalindromicSubsequence;
    

    下面是一些有趣的测试案例, longestPalindromicSubsequence.test.js :

    const longestPalindromicSubsequence = require('./longestPalindromicSubsequence');
    
    describe('longest palindromic subsequence', () => {
      test('works for aab', () => {
        expect(longestPalindromicSubsequence('aab')).toBe(2);
      });
    
      test('works for long string', () => {
        expect(longestPalindromicSubsequence(`${'a'.repeat(50)}bcdef`)).toBe(50);
      });
    });
    

    这是可行的,但由于递归调用的数量呈指数级增长,速度相当慢。例如,对于长度为~50的字符串,需要9秒:

    $ jest longestPalindromicSubsequence.test.js
     PASS  ./longestPalindromicSubsequence.test.js (9.6s)
      longest palindromic subsequence
        ✓ works for aab (3ms)
        ✓ works for long string (9315ms)
    
    Test Suites: 1 passed, 1 total
    Tests:       2 passed, 2 total
    Snapshots:   0 total
    Time:        10.039s
    Ran all test suites matching /longestPalindromicSubsequence.test.js/i.
    

    _.memoize 在更新的模块中 longestPalindromicSubsequence2.js

    const _ = require('lodash');
    
    const longestPalindromicSubsequence = _.memoize(
      (string, start = 0, end = string.length) => {
        if (end < start) { return 0; }
        if (start === end) { return 1; }
        if (string[start] === string[end]) {
          return 2 + longestPalindromicSubsequence(string, start + 1, end - 1);
        }
        return Math.max(
          longestPalindromicSubsequence(string, start + 1, end),
          longestPalindromicSubsequence(string, start, end - 1),
        );
      },
      (string, start, end) => [string, start, end], // resolver function
    );
    
    module.exports = longestPalindromicSubsequence;
    

    但是,当我尝试使用此模块运行测试时,会出现“Javascript堆内存不足”错误:

    $ jest longestPalindromicSubsequence.test.js
    
     RUNS  ./longestPalindromicSubsequence.test.js
    
    <--- Last few GCs --->
    at[89308:0x104801e00]    15800 ms: Mark-sweep 1379.2 (1401.3) -> 1379.2 (1401.3) MB, 1720.4 / 0.0 ms  (+ 0.0 ms in 5 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 1735 ms) (average mu = 0.128, current mu = 0.057) allocat[89308:0x104801e00]    17606 ms: Mark-sweep 1390.0 (1412.3) -> 1390.0 (1412.3) MB, 1711.7 / 0.0 ms  (+ 0.0 ms in 4 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 1764 ms) (average mu = 0.091, current mu = 0.052) allocat
    
    <--- JS stacktrace --->
    
    ==== JS stack trace =========================================
    
        0: ExitFrame [pc: 0x20b000bdc01d]
    Security context: 0x1c189571e549 <JSObject>
        1: /* anonymous */ [0x1c18f7682201] [/Users/kurtpeek/GoogleDrive/LeetCode/longestPalindromicSubsequence2.js:~14] [pc=0x20b0015cd091](this=0x1c18d38893a1 <JSGlobal Object>,string=0x1c18f7682271 <String[55]: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaabcdef>,start=45,end=45)
        2: memoized [0x1c18f7682309] [/Users/kurtpeek/GoogleDrive/LeetCode/node_...
    
    FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
     1: 0x100037733 node::Abort() [/usr/local/bin/node]
     2: 0x1000378d6 node::FatalTryCatch::~FatalTryCatch() [/usr/local/bin/node]
     3: 0x10018e57b v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
     4: 0x10018e51c v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
     5: 0x1004682ee v8::internal::Heap::UpdateSurvivalStatistics(int) [/usr/local/bin/node]
     6: 0x100469ed7 v8::internal::Heap::CheckIneffectiveMarkCompact(unsigned long, double) [/usr/local/bin/node]
     7: 0x1004675cb v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/usr/local/bin/node]
     8: 0x1004663e6 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/usr/local/bin/node]
     9: 0x10046eafc v8::internal::Heap::AllocateRawWithLigthRetry(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/usr/local/bin/node]
    10: 0x10046eb48 v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/usr/local/bin/node]
    11: 0x10044eb7a v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationSpace) [/usr/local/bin/node]
    12: 0x100634916 v8::internal::Runtime_AllocateInTargetSpace(int, v8::internal::Object**, v8::internal::Isolate*) [/usr/local/bin/node]
    13: 0x20b000bdc01d 
    Abort trap: 6
    

    据我所知 Node.js heap out of memory ,节点的标准内存使用量是1.7GB,我认为这应该足够了。你知道为什么记忆版本不起作用,以及如何修复它吗?

    2 回复  |  直到 6 年前
        1
  •  2
  •   Kurt Peek    6 年前

    我通过将解析器函数从 (string, start, end) => [string, start, end] (string, start, end) => string + start + end

    const _ = require('lodash');
    
    const longestPalindromicSubsequence = _.memoize(
      (string, start = 0, end = string.length) => {
        if (end < start) { return 0; }
        if (start === end) { return 1; }
        if (string[start] === string[end]) {
          return 2 + longestPalindromicSubsequence(string, start + 1, end - 1);
        }
        return Math.max(
          longestPalindromicSubsequence(string, start + 1, end),
          longestPalindromicSubsequence(string, start, end - 1),
        );
      },
      (string, start, end) => string + start + end, // resolver function
    );
    
    module.exports = longestPalindromicSubsequence;
    

    $ jest longestPalindromicSubsequence.test.js
     PASS  ./longestPalindromicSubsequence.test.js
      longest palindromic subsequence
        ✓ works for aab (3ms)
        ✓ works for long string (3ms)
    
    Test Suites: 1 passed, 1 total
    Tests:       2 passed, 2 total
    Snapshots:   0 total
    Time:        1.004s, estimated 10s
    Ran all test suites matching /longestPalindromicSubsequence.test.js/i.
    

    使用字符串作为缓存的键似乎比使用数组更节省内存—也许是因为字符串在Javascript中是不可变的?欢迎对这一改进作出任何解释。

        2
  •  2
  •   aug Ben James    6 年前

    MapCache 它们定义了似乎假定字符串将被传递的字符串。

    但是再看看 documentation Cache 对象,假定它与它们的映射具有相同的接口。

    创建一个函数来记忆func的结果。如果分解器是 提供给记忆函数的参数。默认情况下,第一个 提供给memoized函数的参数用作映射缓存 钥匙。func是通过memoized的这个绑定来调用的

    注意:缓存作为memoized上的cache属性公开 功能。 它的创建可以通过替换 _.memoize.Cache构造函数 其实例实现了clear、delete、get、has和set的Map方法接口。

    WeakMap . 这是我测试的

    const _ = require('lodash');
    
    // override Cache and use WeakMap
    _.memoize.Cache = WeakMap;
    
    const longestPalindromicSubsequence = _.memoize(
      (string, start = 0, end = string.length) => {
        if (end < start) { return 0; }
        if (start === end) { return 1; }
        if (string[start] === string[end]) {
          return 2 + longestPalindromicSubsequence(string, start + 1, end - 1);
        }
        return Math.max(
          longestPalindromicSubsequence(string, start + 1, end),
          longestPalindromicSubsequence(string, start, end - 1),
        );
      },
      (string, start, end) => [string, start, end], // resolver function
    );
    
    module.exports = longestPalindromicSubsequence;
    

    JSON.stringify 如果字符串的各个部分最终发生碰撞,则最终字符串可能相同)