代码之家  ›  专栏  ›  技术社区  ›  Markus

无需跳绳就能提高精神分析能力

preprocessor boost-spirit c++
  •  1
  • Markus  · 技术社区  · 7 年前

    考虑一个预处理器,它将读取原始文本(没有明显的空白或标记)。

    有3条规则。

    • resolve_para_entry 应该解决调用中的参数。顶级文本作为字符串返回。

    • resolve_para 应该解析整个参数列表,并将所有顶级参数放在字符串列表中。

    • resolve 是条目

    在跟踪迭代器并获取文本部分的过程中

    样品:

    • sometext(para) 期望 para

    • sometext(para1,para2) 预料 para1 和 para2 在字符串列表中

    • sometext(call(a)) 预料 call(a) 在字符串列表中

    • sometext(call(a,b)) 在这里它失败了;很明显,“!lit(',')”不会将解析器带到外部。。

    规则: 

    resolve_para_entry = +(  
     (iter_pos >> lit('(') >> (resolve_para_entry | eps) >> lit(')') >> iter_pos) [_val=  phoenix::bind(&appendString, _val, _1,_3)]
     | (!lit(',') >> !lit(')') >> !lit('(') >> (wide::char_ | wide::space))         [_val = phoenix::bind(&appendChar, _val, _1)]
    );

resolve_para = (lit('(') >> lit(')'))[_val = std::vector<std::wstring>()]  // empty para -> old style
    | (lit('(') >> resolve_para_entry >> *(lit(',') >> resolve_para_entry) > lit(')'))[_val = phoenix::bind(&appendStringList, _val, _1, _2)]
    | eps;
  ;

resolve = (iter_pos >> name_valid >> iter_pos >> resolve_para >> iter_pos);

    最后似乎不是很优雅。也许有一种更好的方法可以在没有skipper的情况下解析这些内容

    1 回复  |  直到 7 年前
        1
  •  1
  •   sehe    7 年前

    事实上,这应该简单得多。

    首先,我不明白为什么船长缺席 完全 相关的

    其次,暴露原始输入最好使用 qi::raw[] 而不是跳舞 iter_pos 和笨拙的语义动作。

    • negating a charset 已完成 ~ ,例如。 ~char_(",()")
    • (p|eps) 拼写会更好 -p
    • (lit('(') >> lit(')')) 可能只是 "()" (毕竟,没有船长,对吧)
    • p >> *(',' >> p) p % ','

    • 鉴于上述情况, resolve_para 简化为: 

      resolve_para = '(' >> -(resolve_para_entry % ',') >> ')';

    • resolve_para_entry 我觉得很奇怪。似乎任何嵌套的括号都被简单地吞掉了。为什么不真正解析递归语法以便检测语法错误?

    以下是我的看法:

    我更愿意将此作为第一步,因为它有助于我思考解析器产品: 

    namespace Ast {

    using ArgList = std::list<std::string>;

    struct Resolve {
        std::string name;
        ArgList arglist;
    };

    using Resolves = std::vector<Resolve>;
}


    qi::rule<It, Ast::Resolves()> start;
qi::rule<It, Ast::Resolve()>  resolve;
qi::rule<It, Ast::ArgList()>  arglist;
qi::rule<It, std::string()>   arg, identifier;

    及其定义: 

    identifier = char_("a-zA-Z_") >> *char_("a-zA-Z0-9_");

arg        = raw [ +('(' >> -arg >> ')' | +~char_(",)(")) ];
arglist    = '(' >> -(arg % ',') >> ')';
resolve    = identifier >> arglist;

start      = *qr::seek[hold[resolve]];

    笔记:

    • 不再有语义动作
    • 没有更多 eps
    • 没有更多 iter_位置

    • 我选择了 arglist 非可选。如果您真的想要,请将其更改回: 

      resolve    = identifier >> -arglist;

      但在我们的示例中,它将产生大量噪声输出。

    • 当然是你的切入点( start iter_位置 您已经在使用的): seek[]

    • 因此暂停: boost::spirit::qi duplicate parsing on the output -在实际的解析器中可能不需要它。

    Live On Coliru 

    #include <boost/fusion/include/adapt_struct.hpp>
#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/repository/include/qi_seek.hpp>

namespace Ast {

    using ArgList = std::list<std::string>;

    struct Resolve {
        std::string name;
        ArgList arglist;
    };

    using Resolves = std::vector<Resolve>;
}

BOOST_FUSION_ADAPT_STRUCT(Ast::Resolve, name, arglist)

namespace qi = boost::spirit::qi;
namespace qr = boost::spirit::repository::qi;

template <typename It>
struct Parser : qi::grammar<It, Ast::Resolves()>
{
    Parser() : Parser::base_type(start) {
        using namespace qi;

        identifier = char_("a-zA-Z_") >> *char_("a-zA-Z0-9_");

        arg        = raw [ +('(' >> -arg >> ')' | +~char_(",)(")) ];
        arglist    = '(' >> -(arg % ',') >> ')';
        resolve    = identifier >> arglist;

        start      = *qr::seek[hold[resolve]];
    }
  private:
    qi::rule<It, Ast::Resolves()> start;
    qi::rule<It, Ast::Resolve()>  resolve;
    qi::rule<It, Ast::ArgList()>  arglist;
    qi::rule<It, std::string()>   arg, identifier;
};

#include <iostream>

int main() {
    using It = std::string::const_iterator;
    std::string const samples = R"--(
Samples:

sometext(para)        → expect para in the string list
sometext(para1,para2) → expect para1 and para2 in string list
sometext(call(a))     → expect call(a) in the string list
sometext(call(a,b))   ← here it fails; it seams that the "!lit(',')" wont make the parser step outside
)--";
    It f = samples.begin(), l = samples.end();

    Ast::Resolves data;
    if (parse(f, l, Parser<It>{}, data)) {
        std::cout << "Parsed " << data.size() << " resolves\n";

    } else {
        std::cout << "Parsing failed\n";
    }

    for (auto& resolve: data) {
        std::cout << " - " << resolve.name << "\n   (\n";
        for (auto& arg : resolve.arglist) {
            std::cout << "       " << arg << "\n";
        }
        std::cout << "   )\n";
    }
}


    Parsed 6 resolves
 - sometext
   (
       para
   )
 - sometext
   (
       para1
       para2
   )
 - sometext
   (
       call(a)
   )
 - call
   (
       a
   )
 - call
   (
       a
       b
   )
 - lit
   (
       '
       '
   )

    更多想法

    最后一个输出显示了当前语法的问题: lit(',') 显然不应被视为具有两个参数的调用。

    最近,我回答了使用参数提取(嵌套)函数调用的问题,这可以更灵活地完成任务:

    奖金

    奖金版本使用 string_view 并且还显示了所有提取词的精确行/列信息。

    注意,它仍然不需要 任何 phoenix或语义动作。相反,它只是定义了要分配给的必要特征 boost::string_view 从迭代器范围。

    Live On Coliru 

    #include <boost/fusion/include/adapt_struct.hpp>
#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/repository/include/qi_seek.hpp>
#include <boost/utility/string_view.hpp>

namespace Ast {

    using Source  = boost::string_view;
    using ArgList = std::list<Source>;

    struct Resolve {
        Source name;
        ArgList arglist;
    };

    using Resolves = std::vector<Resolve>;
}

BOOST_FUSION_ADAPT_STRUCT(Ast::Resolve, name, arglist)

namespace boost { namespace spirit { namespace traits {
    template <typename It>
    struct assign_to_attribute_from_iterators<boost::string_view, It, void> {
        static void call(It f, It l, boost::string_view& attr) { 
            attr = boost::string_view { f.base(), size_t(std::distance(f.base(),l.base())) };
        }
    };
} } }

namespace qi = boost::spirit::qi;
namespace qr = boost::spirit::repository::qi;

template <typename It>
struct Parser : qi::grammar<It, Ast::Resolves()>
{
    Parser() : Parser::base_type(start) {
        using namespace qi;

        identifier = raw [ char_("a-zA-Z_") >> *char_("a-zA-Z0-9_") ];

        arg        = raw [ +('(' >> -arg >> ')' | +~char_(",)(")) ];
        arglist    = '(' >> -(arg % ',') >> ')';
        resolve    = identifier >> arglist;

        start      = *qr::seek[hold[resolve]];
    }
  private:
    qi::rule<It, Ast::Resolves()> start;
    qi::rule<It, Ast::Resolve()>  resolve;
    qi::rule<It, Ast::ArgList()>  arglist;
    qi::rule<It, Ast::Source()>   arg, identifier;
};

#include <iostream>

struct Annotator {
    using Ref = boost::string_view;

    struct Manip {
        Ref fragment, context;

        friend std::ostream& operator<<(std::ostream& os, Manip const& m) {
            return os << "[" << m.fragment << " at line:" << m.line() << " col:" << m.column() << "]";
        }

        size_t line() const {
            return 1 + std::count(context.begin(), fragment.begin(), '\n');
        }
        size_t column() const {
            return 1 + (fragment.begin() - start_of_line().begin());
        }
        Ref start_of_line() const {
            return context.substr(context.substr(0, fragment.begin()-context.begin()).find_last_of('\n') + 1);
        }
    };

    Ref context;
    Manip operator()(Ref what) const { return {what, context}; }
};

int main() {
    using It = std::string::const_iterator;
    std::string const samples = R"--(Samples:

sometext(para)        → expect para in the string list
sometext(para1,para2) → expect para1 and para2 in string list
sometext(call(a))     → expect call(a) in the string list
sometext(call(a,b))   ← here it fails; it seams that the "!lit(',')" wont make the parser step outside
)--";
    It f = samples.begin(), l = samples.end();

    Ast::Resolves data;
    if (parse(f, l, Parser<It>{}, data)) {
        std::cout << "Parsed " << data.size() << " resolves\n";

    } else {
        std::cout << "Parsing failed\n";
    }

    Annotator annotate{samples};

    for (auto& resolve: data) {
        std::cout << " - " << annotate(resolve.name) << "\n   (\n";
        for (auto& arg : resolve.arglist) {
            std::cout << "       " << annotate(arg) << "\n";
        }
        std::cout << "   )\n";
    }
}

    打印 

    Parsed 6 resolves
 - [sometext at line:3 col:1]
   (
       [para at line:3 col:10]
   )
 - [sometext at line:4 col:1]
   (
       [para1 at line:4 col:10]
       [para2 at line:4 col:16]
   )
 - [sometext at line:5 col:1]
   (
       [call(a) at line:5 col:10]
   )
 - [call at line:5 col:34]
   (
       [a at line:5 col:39]
   )
 - [call at line:6 col:10]
   (
       [a at line:6 col:15]
       [b at line:6 col:17]
   )
 - [lit at line:6 col:62]
   (
       [' at line:6 col:66]
       [' at line:6 col:68]
   )

    ¹ Boost Spirit: "Semantic actions are evil"?

    推荐文章
    Pablo  ·  为什么在boost spirit中使用流会严重影响性能?
    7 年前
    Pablo  ·  在一个单过程解析器中有几个匹配项?
    7 年前
    Addy  ·  Boost Karma:使用Boost::optional编写规则
    7 年前
    joshu  ·  益气解析虚字
    7 年前
    Aleph0  ·  切换到更高版本的Boost 1.6.1时出现编译错误
    7 年前
    Carbon  ·  我可以对继承的东西使用BOOST\u FUSION\u ADAPT\u STRUCT吗?
    7 年前
    sofname  ·  升压::精神::气和升压::凤凰::推回
    7 年前
    Markus  ·  无需跳绳就能提高精神分析能力
    7 年前
    Jean-Michaël Celerier  ·  精神上的递归规则。X3
    7 年前
    Filippo  ·  Boost Spirit x3:解析为结构
    7 年前
    关于  
    代码之家 - 一站式码农服务社区
    沪ICP备11025650号