tags and huge external lib
-
- CodeLite Guru
- Posts: 351
- Joined: Mon Oct 20, 2008 7:26 pm
- Genuine User: Yes
- IDE Question: C++
- Location: France
- Contact:
tags and huge external lib
Hi Eran
I wanted to test boost.spirit... and as I defined boost in my include file for tags, it begans to parse it.
It's awfully long (I started to retag workspace 2 hours ago ).
I tried to remove boost from the includes path, but the current parsing continue.
The problem is that I can't kill codelite_indexer.exe in the Windows task manager. I'll have to wait...
I don't know if there is a solution to fix the 'to big lib' problem (tell CodeLite to stop retag if it takes to much time).
Just to let you know about this...
I wanted to test boost.spirit... and as I defined boost in my include file for tags, it begans to parse it.
It's awfully long (I started to retag workspace 2 hours ago ).
I tried to remove boost from the includes path, but the current parsing continue.
The problem is that I can't kill codelite_indexer.exe in the Windows task manager. I'll have to wait...
I don't know if there is a solution to fix the 'to big lib' problem (tell CodeLite to stop retag if it takes to much time).
Just to let you know about this...
Jérémie
- eranif
- CodeLite Plugin
- Posts: 6375
- Joined: Wed Feb 06, 2008 9:29 pm
- Genuine User: Yes
- IDE Question: C++
- Contact:
Re: tags and huge external lib
boost is not the problem. The problem is 2 files ... in boost: vector250 and vector100 each generates tens of thousands entries...
(both files are auto-generated)
On my machine, I have boost in the path, and it parses it un-noticable
But I am using the more standard headers: boost/shared_ptr.hpp and the like.
Can you give me an example of include files to add to my project so it will start parse like crazy?
EDIT: I tried to use the example code from wikipedia to boost.spirit and codelite parsed the files in < second
this is the code I was using:
Eran
(both files are auto-generated)
On my machine, I have boost in the path, and it parses it un-noticable
But I am using the more standard headers: boost/shared_ptr.hpp and the like.
Can you give me an example of include files to add to my project so it will start parse like crazy?
EDIT: I tried to use the example code from wikipedia to boost.spirit and codelite parsed the files in < second
this is the code I was using:
Code: Select all
#include <boost/spirit.hpp>
#include <boost/spirit/actor.hpp>
#include <string>
#include <iostream>
using namespace std;
using namespace boost::spirit;
int main()
{
string input;
cout << "Input a line.\n";
getline(cin, input);
cout << "Got '" << input << "'.\n";
unsigned count = 0;
/*
Next line parses the input (input.c_str()),
using a parser constructed with the following semantics
(identation matches source for clarity):
Zero or more occurrences of (
literal string "cat" ( when matched, increment the counter "count" )
or any character (to move on finding the next occurrence of "cat")
)
*/
parse(input.c_str(),
*( str_p("cat") [ increment_a(count) ]
| anychar_p
));
/*
The parser is constructed by the compiler using operator
overloading and template matching, so the actual work is
done within spirit::parse(), and the expression starting
with * only initializes the rule object that the parse
function uses.
*/
// last, show results.
cout << "The input had " << count
<< " occurrences of 'cat'\n";
return 0;
}
Make sure you have read the HOW TO POST thread
-
- CodeLite Guru
- Posts: 351
- Joined: Mon Oct 20, 2008 7:26 pm
- Genuine User: Yes
- IDE Question: C++
- Location: France
- Contact:
Re: tags and huge external lib
Hi Eran
Parsing is finished now, and the workspace database is 115 Mo (310 files and 76000 tags, which doesn't seems so huge).
Here is the file :
Parsing is finished now, and the workspace database is 115 Mo (310 files and 76000 tags, which doesn't seems so huge).
Here is the file :
Code: Select all
#include <boost/config/warning_disable.hpp>
#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/include/phoenix_core.hpp>
#include <boost/spirit/include/phoenix_operator.hpp>
#include <iostream>
#include <string>
#include <complex>
///////////////////////////////////////////////////////////////////////////////
// Our complex number parser/compiler
///////////////////////////////////////////////////////////////////////////////
//[tutorial_complex_number
namespace client
{
template <typename Iterator>
bool parse_complex(Iterator first, Iterator last, std::complex<double>& c)
{
using boost::spirit::qi::double_;
using boost::spirit::qi::_1;
using boost::spirit::qi::phrase_parse;
using boost::spirit::ascii::space;
using boost::phoenix::ref;
double rN = 0.0;
double iN = 0.0;
bool r = phrase_parse(first, last,
// Begin grammar
(
'(' >> double_[ref(rN) = _1]
>> -(',' >> double_[ref(iN) = _1]) >> ')'
| double_[ref(rN) = _1]
),
// End grammar
space);
if (!r || first != last) // fail if we did not get a full match
return false;
c = std::complex<double>(rN, iN);
return r;
}
}
//]
////////////////////////////////////////////////////////////////////////////
// Main program
////////////////////////////////////////////////////////////////////////////
int
main()
{
std::cout << "/////////////////////////////////////////////////////////\n\n";
std::cout << "\t\tA complex number micro parser for Spirit...\n\n";
std::cout << "/////////////////////////////////////////////////////////\n\n";
std::cout << "Give me a complex number of the form r or (r) or (r,i) \n";
std::cout << "Type [q or Q] to quit\n\n";
std::string str;
while (getline(std::cin, str))
{
if (str.empty() || str[0] == 'q' || str[0] == 'Q')
break;
std::complex<double> c;
if (client::parse_complex(str.begin(), str.end(), c))
{
std::cout << "-------------------------\n";
std::cout << "Parsing succeeded\n";
std::cout << "got: " << c << std::endl;
std::cout << "\n-------------------------\n";
}
else
{
std::cout << "-------------------------\n";
std::cout << "Parsing failed\n";
std::cout << "-------------------------\n";
}
}
std::cout << "Bye... :-) \n\n";
return 0;
}
Jérémie
- eranif
- CodeLite Plugin
- Posts: 6375
- Joined: Wed Feb 06, 2008 9:29 pm
- Genuine User: Yes
- IDE Question: C++
- Contact:
Re: tags and huge external lib
Thanks for this file, like I guessed: its those vectorXXX.hpp files which takes too long to parse.
Other than those files, the parsing is the same speed as other files (very reasonable time)
I can think of several fixes here:
- If codelite_indexer does not respond in XXX seconds - kill it (which will cause the current file to be skipped - small price to pay)
- put in the code a quick and ugly hack to avoid parsing these files
- Maybe placing a memory limit (dont allow more than XXX tags from a given file)
What do you think?
Eran
Other than those files, the parsing is the same speed as other files (very reasonable time)
I can think of several fixes here:
- If codelite_indexer does not respond in XXX seconds - kill it (which will cause the current file to be skipped - small price to pay)
- put in the code a quick and ugly hack to avoid parsing these files
- Maybe placing a memory limit (dont allow more than XXX tags from a given file)
What do you think?
Eran
Make sure you have read the HOW TO POST thread
-
- CodeLite Guru
- Posts: 351
- Joined: Mon Oct 20, 2008 7:26 pm
- Genuine User: Yes
- IDE Question: C++
- Location: France
- Contact:
Re: tags and huge external lib
Why not. I'm asking myself what could happen at each workspace retag (or at least when saving the workspace file which include the evil .h file) ? Does the following include files will be parse also, to allow a good code completion anyway ? Are you sure you can kill the process ? From the task manager, I didn't succed !eranif wrote:- If codelite_indexer does not respond in XXX seconds - kill it (which will cause the current file to be skipped - small price to pay)
Arghhhheranif wrote:- put in the code a quick and ugly hack to avoid parsing these files
That will probably decrease parsing performances, because you have to ask for the number of tags in the database during parsing. But that sounds pretty good. We can suppose that this kind of file is not important for code completion, but for internal use of the lib. Moreover, it has the advantage to not trying to retag the evil file each time (unlike the 1st solution).eranif wrote:- Maybe placing a memory limit (dont allow more than XXX tags from a given file)
Jérémie
- eranif
- CodeLite Plugin
- Posts: 6375
- Joined: Wed Feb 06, 2008 9:29 pm
- Genuine User: Yes
- IDE Question: C++
- Contact:
Re: tags and huge external lib
ofc I can. I am doing it already (whenver you are closing the tags settings dialog the indexer process is restarted)jfouche wrote:Are you sure you can kill the process ? From the task manager, I didn't succed !
You may think that the process is not killed, is simply being restarted again by codelite (notice that it has a different PID)
I can get the number of tags from the indexer: the number of tags equals the number of lines returnedjfouche wrote:That will probably decrease parsing performances, because you have to ask for the number of tags in the database during parsing
Eran
Make sure you have read the HOW TO POST thread
-
- CodeLite Guru
- Posts: 351
- Joined: Mon Oct 20, 2008 7:26 pm
- Genuine User: Yes
- IDE Question: C++
- Location: France
- Contact:
Re: tags and huge external lib
Hi Eran
I think the 3rd solution is the better one, mostly because it will not try to parse the file each time. I just would like to be sure the parsing of the evil file can stop even if the file is not fully parsed (does lex allow it) ?
BTW : I thought CL developpment was near to be finished... But the TODO list continue. Do you want me to add a feature request on sourceforge ? It would great to have a roadmap based on the sourceforge bugs / requests (like TRAC). The one on the CodeLite web site is out f date now (2.0 is released). I don't think sourceforge provide something like this. Never mind...
I think the 3rd solution is the better one, mostly because it will not try to parse the file each time. I just would like to be sure the parsing of the evil file can stop even if the file is not fully parsed (does lex allow it) ?
BTW : I thought CL developpment was near to be finished... But the TODO list continue. Do you want me to add a feature request on sourceforge ? It would great to have a roadmap based on the sourceforge bugs / requests (like TRAC). The one on the CodeLite web site is out f date now (2.0 is released). I don't think sourceforge provide something like this. Never mind...
Jérémie
- eranif
- CodeLite Plugin
- Posts: 6375
- Joined: Wed Feb 06, 2008 9:29 pm
- Genuine User: Yes
- IDE Question: C++
- Contact:
Re: tags and huge external lib
I think that SF provides TRAC, I need to enable it. If I will enable it, will you be able to maintain it?
Eran
Eran
Make sure you have read the HOW TO POST thread
-
- CodeLite Guru
- Posts: 351
- Joined: Mon Oct 20, 2008 7:26 pm
- Genuine User: Yes
- IDE Question: C++
- Location: France
- Contact:
Re: tags and huge external lib
Yes, I can do this. I just have to beware of your commits (with CommitMonitor, it's easy), and you just have to let me know the feature you want to implement for a given version (I'll probably ask you from time to times). Let's have a try...eranif wrote:If I will enable it, will you be able to maintain it?
Jérémie
- eranif
- CodeLite Plugin
- Posts: 6375
- Joined: Wed Feb 06, 2008 9:29 pm
- Genuine User: Yes
- IDE Question: C++
- Contact:
Re: tags and huge external lib
Hi,
CodeLite Trac can be found here:
https://sourceforge.net/apps/trac/codelite/
I gave you permissions on codelite project (I am not sure they sufficient though) - let me know if you can update the Trac pages
Eran
CodeLite Trac can be found here:
https://sourceforge.net/apps/trac/codelite/
I gave you permissions on codelite project (I am not sure they sufficient though) - let me know if you can update the Trac pages
Eran
Make sure you have read the HOW TO POST thread