
项目背景
在一个庞大的代码自动生成项目中,需要通过对C/C++头文件的扫描分析并记录所有public类型的函数原型及参数类型。
在XML script中,显式使用C函数和参数。用户可以自行编写xml测试脚本来运行C/C++代码。
上述过程都通过perl script自动生成并编译。
问题
自动生成代码量巨大,很大部分的时间瓶颈在于perl脚本的执行时间。需要进行优化。
方法
利用了perl的profiling机制进行运行时profiling分析并分perl函数统计所有时间开销。
最后重点改善耗费时间最多的20-30个瓶颈函数。
结果
不包括makefile等方面的脚本编译优化,不考虑perl脚本的功能实现大变动,perl执行时间有大约10~15%的提升。
-------------------------------------------------------------------------------------------------------------------------------------------------------------
了解速度瓶颈
perl -d:DProf test.pl Perl dprofpp 显示perl的概要设置数据。 Devel::DProf运行时增加-d:DProf选项,会自动生成一个tmon.out文件 查看profile结果,运行dprof
典型的,tmon.out示例如下
@H_502_128@#fOrTyTwO$hz=1000;$XS_VERSION='DProf 20050310.00';# All values are given in HZ$over_utime=32; $over_stime=0; $over_rtime=32;$over_tests=10000;$rrun_utime=15; $rrun_stime=0; $rrun_rtime=15;$total_marks=2 PART2@ 15 0 15& 2 main BEGIN+ 2- 2
对于perl脚本执行后产生的profiling概要数据示例如下
Total Elapsed Time = 0.139411 Seconds User+System Time = 0.061411 SecondsExclusive Times%Time ExclSec CumulS #Calls sec/call Csec/c name 26.0 0.016 0.016 1 0.0160 0.0160 warnings::BEGIN 26.0 0.016 0.016 1 0.0160 0.0160 fileutils::RemovefilesWithContent 24.4 0.015 0.015 6 0.0025 0.0025 FindBin::BEGIN 24.4 0.015 0.045 7 0.0021 0.0065 main::BEGIN 0.00 - -0.000 1 - - file::Basename::fileparse_set_fsty 0.0000 pe 0.00 - -0.000 1 - - lib::_nativize 0.00 - -0.000 1 - - Getopt::Long::ConfigDefaults 0.00 - -0.000 1 - - Getopt::Long::Configure 0.00 - -0.000 1 - - DynaLoader::dl_load_file 0.00 - -0.000 1 - - DynaLoader::dl_undef_symbols 0.00 - -0.000 1 - - DynaLoader::dl_find_symbol 0.00 - -0.000 1 - - DynaLoader::dl_install_xsub 0.00 - -0.000 1 - - Cwd::bootstrap 0.00 - -0.000 1 - - Config::TIEHASH 0.00 - -0.000 1 - - re::bits
perl执行时间统计结果示例
====== Final Data Result in Values order ( Time Unit: second) ======134.68317 <= all_total_elapsed_time83.519373 <= all_user_system_time42.560276 <= genentityfile.pl::total_elapsed_time24.133051 <= genAPIIDs.pl::total_elapsed_time18.171 <= main::BEGIN16.265623 <= genAPInamesfromtemplate.pl::total_elapsed_time15.991419 <= genAPInamesfromtemplate.pl::user_system_time10.250529 <= genentityfile.pl::user_system_time10.030177 <= genmoduleAPIfiles.pl::total_elapsed_time9.902177 <= genmoduleAPIfiles.pl::user_system_time8.354992 <= genscriptschema.pl::total_elapsed_time8.294992 <= genscriptschema.pl::user_system_time7.45852 <= genmoduLeversionfile.pl::total_elapsed_time7.417362 <= genAPIIDs.pl::user_system_time7.36752 <= genmoduLeversionfile.pl::user_system_time7.227387 <= convertxscriptAPIs.pl::total_elapsed_time7.136385 <= convertxscriptAPIs.pl::user_system_time6.909191 <= genatsfile.pl::total_elapsed_time6.868048 <= genatsfile.pl::user_system_time5.80599999999999 <= deBUGutils::BEGIN5.69099999999999 <= IO::file5.67999999999999 <= file::Temp4.112021 <= compilescript.pl::total_elapsed_time4.112 <= FindBin::BEGIN4.09702 <= compilescript.pl::user_system_time3.639 <= tmpfile::BEGIN3.589 <= templateparser::BEGIN3.19 <= datamanager::BEGIN3.089 <= IO::Seekable2.902 <= Benchmark::BEGIN2.451 <= templateparser::ReplacenameValuePa2.403 <= vars::BEGIN2.24 <= Getopt::Long2.159 <= enumparser::ParseEnumfile1.867215 <= geneventhandlerfromtemplate.pl::total_elapsed_time1.867205 <= geneventhandlerfromtemplate.pl::user_system_time1.609 <= IO::Handle1.387 <= Exporter::as_heavy1.35 <= APIsparser::ParseAPIs1.295 <= warnings::BEGIN1.133 <= DynaLoader::dl_load_file1.05 <= configparser::BEGIN0.943427 <= genfilefromtemplate.pl::total_elapsed_time0.908427 <= genfilefromtemplate.pl::user_system_time0.846000000000001 <= cpreprocessfile::Preprocessfile0.827000000000001 <= file::Basename0.809 <= Exporter::import0.798000000000001 <= APIsparser::BEGIN0.74 <= calculatecrc::CRC16_CalculateCRC0.729000000000001 <= scriptAPIparser::BEGIN0.690000000000001 <= DynaLoader::bootstrap0.681 <= vars::import0.642 <= eventsparser::ParseEvents0.598 <= Storable::BEGIN0.573 <= templateparser::StripSubstringForI0.560688 <= genglobalrefsfromtemplate.pl::user_system_time0.559688 <= genglobalrefsfromtemplate.pl::total_elapsed_time0.532 <= enumparser::BEGIN0.482 <= XSLoader::load0.471 <= autoLoader::autoLOAD0.465705 <= findfiles.pl::total_elapsed_time0.436 <= Cwd::fast_abs_path0.41 <= templateparser::PreserveSubstringF0.392 <= APIdeffileparser::GetGlobalsForMod0.389 <= enumparser::GetDigitValue0.368973 <= genentityversionfile.pl::total_elapsed_time0.368973 <= genentityversionfile.pl::user_system_time0.357 <= isiconfigparser::BEGIN0.353 <= deBUGutils::DeBUGPrint0.343 <= lib::import0.331 <= Storable::pretrIEve0.303 <= APIsparser::GetParentStructurename0.295134 <= genxmlmultiinput.pl::total_elapsed_time0.285 <= APIsparser::GetnamespaceFromAPIIDS0.279134 <= genxmlmultiinput.pl::user_system_time0.279 <= file::copy0.277984 <= genadaptergroups.pl::total_elapsed_time0.277984 <= genadaptergroups.pl::user_system_time0.255 <= APIsparser::GetStructsFromline0.24919 <= SCRIPTS_ListS_DIAG::total_elapsed_time0.231705 <= findfiles.pl::user_system_time0.231 <= templateparser::Processtemplate0.219 <= cpreprocessfile::StripPreprocessor0.217 <= constant::import0.215 <= valIDateparams::ValIDatePos0.207 <= warnings::register0.206 <= entityparser::ParseEntityfile0.204 <= cpreprocessfile::BEGIN0.188 <= file::stat0.187 <= moduleIDsparser::BEGIN0.173 <= storabledata::BEGIN0.171205 <= SCRIPTS_INC_SRCS_DIAG::total_elapsed_time0.147 <= templateparser::ReplaceIfCondition0.141 <= main::WritefileContents_NoXmlVersi0.138 <= adaptercodegen::WriteForAPI0.131 <= APIsparser::IsParamNotScriptableBy0.124 <= main::Writefile_NoXmlVersion0.124 <= scriptcompiler::BEGIN0.123 <= FindBin::init0.11 <= main::WriteData0.10843 <= getxscriptversion.pl::total_elapsed_time0.105502 <= geneventhandlerfactoryfromtemplate.pl::user_system_time0.105502 <= geneventhandlerfactoryfromtemplate.pl::total_elapsed_time0.104 <= scriptAPIparser::GetCppClassnames0.099 <= templateparser::StripSubstring0.095 <= tmpfile::DeleteTmpfile0.093205 <= SCRIPTS_INC_SRCS_DIAG::user_system_time0.093 <= file::Spec0.09219 <= SCRIPTS_ListS_DIAG::user_system_time0.085 <= enumparser::GetEnumValueFromMaps0.083 <= main::WriteForEachEntityfile0.08 <= autoLoader::import0.079 <= datamanager::OutputnameValuePair0.079 <= main::WriteDefs0.079 <= strict::unimport0.078 <= scriptAPIparser::GetModuleList0.078 <= main::ProcessXml0.076 <= pounddefineparser::ParsedefinesFil0.074 <= templateparser::CreateEnumnameFrom0.074 <= templateparser::PreserveSubstring0.073 <= main::WriteVersions0.07 <= Regexp::DESTROY0.063 <= main::EnumGroupCallback0.063 <= Storable::bootstrap0.063 <= DynaLoader::dl_install_xsub0.062 <= strict::import0.062 <= datamanager::WriteSetnameValuePair0.06143 <= getxscriptversion.pl::user_system_time0.061 <= Class::Struct0.06 <= IO::bootstrap0.047 <= warnings::import0.047 <= eventhandlerparser::BEGIN0.047 <= err::BEGIN0.047 <= main::GetGlobalVariableList0.046 <= main::WriteScriptSymbolsForAPI0.046 <= datamanager::WriteFIEld0.046 <= strict::bits0.046 <= overload::OVERLOAD0.046 <= enumparser::ParseEnumfileWithPrepr0.046 <= APIsparser::ReplaceXmlEntitIEsInCp0.046 <= scriptAPIparser::ConvertScriptfile0.046 <= eventhandlerparser::GetEventHandle0.045 <= Time::HiRes0.045 <= datamanager::ResolveFIEldname0.032 <= enumpreserver::GetExistingEnumValu0.031 <= main::WriteSchemaForLoopCB0.031 <= adaptercodegen::WriteHandleResultF0.031 <= main::WriteEntityParamIDsForAPI0.031 <= fileutils::BEGIN0.031 <= APIsparser::GetArgsArray0.031 <= fileutils::ExecShellFindCmd0.031 <= templateparser::WriteForEachInclud0.031 <= APIsparser::HasFIEldsWithUseType0.031 <= Fcntl::constant0.031 <= datamanager::OpenIfs0.031 <= main::WriteDefsParamsForAPI0.03 <= APIsparser::IsArrayElement0.016 <= main::WriteFunctionDefsForAPI0.016 <= main::GetModuleSchemafilename0.016 <= APIsparser::GetReturnname0.016 <= Fcntl::autoLOAD0.016 <= re::import0.016 <= datamanager::ConvertToDataMgrEntit0.016 <= Symbol::gensym0.016 <= main::WriteEntitIEsForEvent0.016 <= versionparser::BEGIN0.016 <= valIDateparams::BEGIN0.016 <= SelectSaver::new0.016 <= datamanager::GenPtrStructCppExpr0.016 <= storabledata::GetDataForPhase30.016 <= enumparser::CallEnumGroupCallback0.016 <= APIsparser::IsDescendant0.016 <= Config::import0.016 <= valIDateparams::ValIDate0.016 <= APIsparser::GetDataItemFromList0.016 <= main::WriteOutputEntitIEsForAPI0.016 <= main::CompileScript0.016 <= main::GetAPIsForModule0.016 <= eventsparser::GetPayloadnameAndTyp0.016 <= main::WriteAtsForLoopCB0.016 <= Benchmark::init0.016 <= Storable::_retrIEve0.016 <= enumpreserver::copyfileUntilEndMar0.016 <= APIsparser::GetParam0.015 <= fileutils::RemovefilesWithContent0.015 <= main::WriteFooterForSchema0.015 <= autoLoader::BEGIN0.015 <= IO::BEGIN0.015 <= APIsparser::IsUnionorVoIDPtrDescen0.015 <= main::WriteSchemaForAPI0.015 <= datamanager::GetClosestAncestorStr0.015 <= datamanager::CloseForLoops0.015 <= Cwd::cwd0.015 <= scriptAPIparser::GetAPIFromScriptL0.015 <= main::WriteDataParamsForAPI0.015 <= enumpreserver::WriteEnum0.015 <= base::has_version7.9e-05 <= convertmoduleIDs.pl::total_elapsed_time7.8e-05 <= convertmoduleIDs.pl::user_system_time0 <= Storable::_store0 <= Exporter::Heavy0 <= Storable::retrIEve0 <= Storable::store0 <= Fcntl::__ANON__总结
以上是内存溢出为你收集整理的Perl profiling笔记全部内容,希望文章能够帮你解决Perl profiling笔记所遇到的程序开发问题。
如果觉得内存溢出网站内容还不错,欢迎将内存溢出网站推荐给程序员好友。
欢迎分享,转载请注明来源:内存溢出
微信扫一扫
支付宝扫一扫
评论列表(0条)