好的,在发布的代码中,70%的时间花在流的下溢操作上。
我
没有调查/为什么/那是,但是
写了一些天真的实现,看看我是否可以做得更好。第一步:
²
使现代化
我从那以后
analyzed it
并提供了
PR
。
在这种特殊情况下,该PR产生的改进不会影响底线(参见
总结
)
-
滴
operator>>
对于
Timestamp
(我们不会使用它)
-
替换的所有实例
'[' >> stream >> ']'
使用替代方案
'[' >> raw[*~char_(']')] >> ']'
因此,我们将始终使用trait将迭代器范围转换为属性类型(
std::string
或
时间戳
)
现在,我们实现
assign_to_attribute_from_iterators<structs::Timestamp, It>
特点:
变体1:阵列源
template <typename It>
struct assign_to_attribute_from_iterators<structs::Timestamp, It, void> {
static inline void call(It f, It l, structs::Timestamp& time) {
boost::iostreams::stream<boost::iostreams::array_source> stream(f, l);
struct std::tm tm;
if (stream >> std::get_time(&tm, "%Y-%b-%d %H:%M:%S") >> time.ms)
time.date = std::mktime(&tm);
else throw "Parse failure";
}
};
使用callgrind进行评测:
(单击可缩放)
它确实有了很大的改进,可能是因为我们假设底层的char缓冲区是连续的,而Spirit实现无法做出这种假设。我们花了大约42%的时间在
time_get
。
粗略地说,25%的时间都花在现场工作上,其中令人担忧的是
约20%
用于执行动态强制转换:(
变体2:重复使用的阵列源
相同,但重用静态流实例以查看其是否有显著差异:
static boost::iostreams::stream<boost::iostreams::array_source> s_stream;
template <typename It>
struct assign_to_attribute_from_iterators<structs::Timestamp, It, void> {
static inline void call(It f, It l, structs::Timestamp& time) {
struct std::tm tm;
if (s_stream.is_open()) s_stream.close();
s_stream.clear();
boost::iostreams::array_source as(f, l);
s_stream.open(as);
if (s_stream >> std::get_time(&tm, "%Y-%b-%d %H:%M:%S") >> time.ms)
time.date = std::mktime(&tm);
else throw "Parse failure";
}
};
分析显示没有显著差异)。
变体3:
strptime
和
strtod
/
from_chars
让我们看看降到C级是否会减少对语言环境的伤害:
template <typename It>
struct assign_to_attribute_from_iterators<structs::Timestamp, It, void> {
static inline void call(It f, It l, structs::Timestamp& time) {
struct std::tm tm;
auto remain = strptime(&*f, "%Y-%b-%d %H:%M:%S", &tm);
time.date = std::mktime(&tm);
#if __has_include(<charconv>) || __cpp_lib_to_chars >= 201611
auto result = std::from_chars(&*f, &*l, time.ms); // using <charconv> from c++17
#else
char* end;
time.ms = std::strtod(remain, &end);
assert(end > remain);
static_cast<void>(l); // unused
#endif
}
};
如您所见,使用
字符串转换为浮点数
这里有点不太理想。输入范围是有界的,但无法判断
字符串转换为浮点数
关于这个。我无法分析
来自\u chars
这种方法更安全,因为它没有这个问题。
在实践中,对于示例代码,使用它是安全的
字符串转换为浮点数
因为我们知道输入缓冲区是NUL终止的。
在这里,您可以看到解析日期时间仍然是一个值得关注的因素:
-
mktime 15.58%
-
STRTIME 40.54%
-
标准差5.88%
但总的来说,现在的差别不那么严重了:
-
解析器1:14.17%
-
解析器2:43.44%
-
Parser3:5.69%
-
解析器4:35.49%
变体4:再次启动日期时间
有趣的是,“低级别”C-API的性能离使用更高级别的Boost不远了
posix_time::ptime
功能:
template <typename It>
struct assign_to_attribute_from_iterators<structs::Timestamp, It, void> {
static inline void call(It f, It l, structs::Timestamp& time) {
time.date = to_time_t(boost::posix_time::time_from_string(std::string(f,l)));
}
};
这可能会牺牲一些精度,
according to the docs
:
在这里,解析日期和时间所花费的总时间是68%。解析器的相对速度接近最后的速度:
-
解析器1:12.33%
-
解析器2:43.86%
-
解析器3:5.22%
-
解析器4:37.43%
总结
总之,结果表明,存储字符串似乎更快,即使您有可能分配更多字符串。我做了一个非常简单的检查
SSO
通过增加子字符串的长度:
static const std::string input1 = "[2018-Mar-01 00:01:02.012345 THWARTING THE SMALL STRING OPTIMIZATION HERE THIS WON'T FIT, NO DOUBT] - 1.000 s => String: Valid_string\n";
static const std::string input2 = "[2018-Mar-02 00:01:02.012345 THWARTING THE SMALL STRING OPTIMIZATION HERE THIS WON'T FIT, NO DOUBT] - 2.000 s => I dont care\n";
没有重大影响,因此只剩下解析本身。
很明显,要么您想要延迟解析时间(
Parser3
是目前为止最快的)
或
应该与久经考验的助力相匹配
posix_time
功能。
表册
下面是我使用的组合基准代码。一些事情发生了变化:
-
添加了一些健全性检查输出(以避免测试无意义的代码)
-
使迭代器成为泛型(更改为
char*
对优化构建中的性能没有显著影响)
-
上述变量都可以通过更改
#if 1
到
#if 0
在正确的地点
-
为方便起见,减少了N1/N2
我经常使用C++14,因为代码的目的是找到瓶颈。在分析之后,获得的任何智慧都可以相对容易地进行后传。
Live On Coliru
#include <boost/fusion/adapted/struct.hpp>
#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/include/phoenix.hpp>
#include <boost/spirit/repository/include/qi_seek.hpp>
#include <boost/date_time/posix_time/posix_time.hpp>
#include <boost/chrono/chrono.hpp>
#include <iomanip>
#include <ctime>
#if __has_include(<charconv>) || __cpp_lib_to_chars >= 201611
# include <charconv> // not supported yet until GCC 8
#endif
namespace structs {
struct Timestamp {
std::time_t date;
double ms;
};
struct Record1 {
std::string date;
double time;
std::string str;
};
struct Record2 {
Timestamp date;
double time;
std::string str;
};
typedef std::vector<Record1> Records1;
typedef std::vector<Record2> Records2;
}
BOOST_FUSION_ADAPT_STRUCT(structs::Record1,
(std::string, date)
(double, time)
(std::string, str))
BOOST_FUSION_ADAPT_STRUCT(structs::Record2,
(structs::Timestamp, date)
(double, time)
(std::string, str))
namespace boost { namespace spirit { namespace traits {
template <typename It>
struct assign_to_attribute_from_iterators<std::string, It, void> {
static inline void call(It f, It l, std::string& attr) {
attr = std::string(&*f, std::distance(f,l));
}
};
static boost::iostreams::stream<boost::iostreams::array_source> s_stream;
template <typename It>
struct assign_to_attribute_from_iterators<structs::Timestamp, It, void> {
static inline void call(It f, It l, structs::Timestamp& time) {
#if 1
time.date = to_time_t(boost::posix_time::time_from_string(std::string(f,l)));
#elif 1
struct std::tm tm;
boost::iostreams::stream<boost::iostreams::array_source> stream(f, l);
if (stream >> std::get_time(&tm, "%Y-%b-%d %H:%M:%S") >> time.ms)
time.date = std::mktime(&tm);
else
throw "Parse failure";
#elif 1
struct std::tm tm;
if (s_stream.is_open()) s_stream.close();
s_stream.clear();
boost::iostreams::array_source as(f, l);
s_stream.open(as);
if (s_stream >> std::get_time(&tm, "%Y-%b-%d %H:%M:%S") >> time.ms)
time.date = std::mktime(&tm);
else
throw "Parse failure";
#else
struct std::tm tm;
auto remain = strptime(&*f, "%Y-%b-%d %H:%M:%S", &tm);
time.date = std::mktime(&tm);
#if __has_include(<charconv>) || __cpp_lib_to_chars >= 201611
auto result = std::from_chars(&*f, &*l, time.ms); // using <charconv> from c++17
#else
char* end;
time.ms = std::strtod(remain, &end);
assert(end > remain);
static_cast<void>(l); // unused
#endif
#endif
}
};
} } }
namespace qi = boost::spirit::qi;
namespace QiParsers {
template <typename It>
struct Parser1 : qi::grammar<It, structs::Record1()>
{
Parser1() : Parser1::base_type(start) {
using namespace qi;
start = '[' >> raw[*~char_(']')] >> ']'
>> " - " >> double_ >> " s"
>> " => String: " >> raw[+graph]
>> eol;
}
private:
qi::rule<It, structs::Record1()> start;
};
template <typename It>
struct Parser2 : qi::grammar<It, structs::Record2()>
{
Parser2() : Parser2::base_type(start) {
using namespace qi;
start = '[' >> raw[*~char_(']')] >> ']'
>> " - " >> double_ >> " s"
>> " => String: " >> raw[+graph]
>> eol;
}
private:
qi::rule<It, structs::Record2()> start;
};
template <typename It>
struct Parser3 : qi::grammar<It, structs::Records1()>
{
Parser3() : Parser3::base_type(start) {
using namespace qi;
using boost::phoenix::push_back;
line = '[' >> raw[*~char_(']')] >> ']'
>> " - " >> double_ >> " s"
>> " => String: " >> raw[+graph];
ignore = *~char_("\r\n");
start = (line[push_back(_val, _1)] | ignore) % eol;
}
private:
qi::rule<It> ignore;
qi::rule<It, structs::Record1()> line;
qi::rule<It, structs::Records1()> start;
};
template <typename It>
struct Parser4 : qi::grammar<It, structs::Records2()>
{
Parser4() : Parser4::base_type(start) {
using namespace qi;
using boost::phoenix::push_back;
line = '[' >> raw[*~char_(']')] >> ']'
>> " - " >> double_ >> " s"
>> " => String: " >> raw[+graph];
ignore = *~char_("\r\n");
start = (line[push_back(_val, _1)] | ignore) % eol;
}
private:
qi::rule<It> ignore;
qi::rule<It, structs::Record2()> line;
qi::rule<It, structs::Records2()> start;
};
}
template <typename Parser> static const Parser s_instance {};
template<template <typename> class Parser, typename Container, typename It>
Container parse_seek(It b, It e, const std::string& message)
{
Container records;
auto const t0 = boost::chrono::high_resolution_clock::now();
parse(b, e, *boost::spirit::repository::qi::seek[s_instance<Parser<It> >], records);
auto const t1 = boost::chrono::high_resolution_clock::now();
auto elapsed = boost::chrono::duration_cast<boost::chrono::milliseconds>(t1 - t0);
std::cout << "Elapsed time: " << elapsed.count() << " ms (" << message << ")\n";
return records;
}
template<template <typename> class Parser, typename Container, typename It>
Container parse_ignoring(It b, It e, const std::string& message)
{
Container records;
auto const t0 = boost::chrono::high_resolution_clock::now();
parse(b, e, s_instance<Parser<It> >, records);
auto const t1 = boost::chrono::high_resolution_clock::now();
auto elapsed = boost::chrono::duration_cast<boost::chrono::milliseconds>(t1 - t0);
std::cout << "Elapsed time: " << elapsed.count() << " ms (" << message << ")\n";
return records;
}
static const std::string input1 = "[2018-Mar-01 00:01:02.012345] - 1.000 s => String: Valid_string\n";
static const std::string input2 = "[2018-Mar-02 00:01:02.012345] - 2.000 s => I dont care\n";
std::string prepare_input() {
std::string input;
const int N1 = 10;
const int N2 = 1000;
input.reserve(N1 * (input1.size() + N2*input2.size()));
for (int i = N1; i--;) {
input += input1;
for (int j = N2; j--;)
input += input2;
}
return input;
}
int main() {
auto const input = prepare_input();
auto f = input.data(), l = f + input.length();
for (auto& r: parse_seek<QiParsers::Parser1, structs::Records1>(f, l, "std::string + seek")) {
std::cout << r.date << "\n";
break;
}
for (auto& r: parse_seek<QiParsers::Parser2, structs::Records2>(f, l, "stream + seek")) {
auto tm = *std::localtime(&r.date.date);
std::cout << std::put_time(&tm, "%Y-%b-%d %H:%M:%S") << "\n";
break;
}
for (auto& r: parse_ignoring<QiParsers::Parser3, structs::Records1>(f, l, "std::string + ignoring")) {
std::cout << r.date << "\n";
break;
}
for (auto& r: parse_ignoring<QiParsers::Parser4, structs::Records2>(f, l, "stream + ignoring")) {
auto tm = *std::localtime(&r.date.date);
std::cout << std::put_time(&tm, "%Y-%b-%d %H:%M:%S") << "\n";
break;
}
}
打印类似的内容
Elapsed time: 14 ms (std::string + seek)
2018-Mar-01 00:01:02.012345
Elapsed time: 29 ms (stream + seek)
2018-Mar-01 00:01:02
Elapsed time: 2 ms (std::string + ignoring)
2018-Mar-01 00:01:02.012345
Elapsed time: 22 ms (stream + ignoring)
2018-Mar-01 00:01:02
所有百分比都是相对于
全部的
项目成本。那个
做
扭曲百分比(如果不考虑非流解析器测试,提到的70%会更糟),但这些数字对于相对比较来说是一个足够好的指南
在内部
试运行。
