<table><tr><td style="">aaronpuchert added a comment.
</td><a style="text-decoration: none; padding: 4px 8px; margin: 0 8px 8px; float: right; color: #464C5C; font-weight: bold; border-radius: 3px; background-color: #F7F7F9; background-image: linear-gradient(to bottom,#fff,#f1f0f1); display: inline-block; border: 1px solid rgba(71,87,120,.2);" href="https://phabricator.kde.org/D18551">View Revision</a></tr></table><br /><div><div><p>To keep this short I didn't reply to the parts I completely agree with.</p>
<blockquote style="border-left: 3px solid #8C98B8;
color: #6B748C;
font-style: italic;
margin: 4px 0 12px 0;
padding: 8px 12px;
background-color: #F8F9FC;">
<div style="font-style: normal;
padding-bottom: 4px;">In <a href="https://phabricator.kde.org/D18551#604249" style="background-color: #e7e7e7;
border-color: #e7e7e7;
border-radius: 3px;
padding: 0 4px;
font-weight: bold;
color: black;text-decoration: none;">D18551#604249</a>, <a href="https://phabricator.kde.org/p/mwolff/" style="
border-color: #f1f7ff;
color: #19558d;
background-color: #f1f7ff;
border: 1px solid transparent;
border-radius: 3px;
font-weight: bold;
padding: 0 4px;">@mwolff</a> wrote:</div>
<div style="margin: 0;
padding: 0;
border: 0;
color: rgb(107, 116, 140);"><blockquote style="border-left: 3px solid #a7b5bf; color: #464c5c; font-style: italic; margin: 4px 0 12px 0; padding: 4px 12px; background-color: #f8f9fc;"><p>Now I actually have <tt style="background: #ebebeb; font-size: 13px;">/tmp</tt> as tmpfs, and so these 400 MB don't come for free. This is perfectly fine when I'm actually working on these files, but so far I have only opened KDevelop.</p></blockquote>
<p>I see. The problem really is that the preamble files shouldn't be written to /tmp then - it would be perfectly fine to move them to a temp dir on the disk I think - it would still be faster than having to redo the work.</p></div>
</blockquote>
<p>There are arguments for both in-memory and on-disk, it depends on how much memory there is. I have another more more generously sized machine where the files are perfectly fine in RAM.</p>
<p>I think <tt style="background: #ebebeb; font-size: 13px;">/tmp</tt> is <a href="https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch03s18.html" class="remarkup-link" target="_blank" rel="noreferrer">correct by the FHS</a> and <tt style="background: #ebebeb; font-size: 13px;">/var/tmp</tt> <a href="https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch05s15.html" class="remarkup-link" target="_blank" rel="noreferrer">wouldn't be appropriate</a>. I could easily put this on disk if I wanted to. I just wanted to make the argument that resource usage is a concern, and that there is some benefit in limiting ourselves to the preambles that we need.</p>
<blockquote style="border-left: 3px solid #a7b5bf; color: #464c5c; font-style: italic; margin: 4px 0 12px 0; padding: 4px 12px; background-color: #f8f9fc;"><blockquote style="border-left: 3px solid #a7b5bf; color: #464c5c; font-style: italic; margin: 4px 0 12px 0; padding: 4px 12px; background-color: #f8f9fc;"><blockquote style="border-left: 3px solid #a7b5bf; color: #464c5c; font-style: italic; margin: 4px 0 12px 0; padding: 4px 12px; background-color: #f8f9fc;"><blockquote style="border-left: 3px solid #a7b5bf; color: #464c5c; font-style: italic; margin: 4px 0 12px 0; padding: 4px 12px; background-color: #f8f9fc;"><blockquote style="border-left: 3px solid #a7b5bf; color: #464c5c; font-style: italic; margin: 4px 0 12px 0; padding: 4px 12px; background-color: #f8f9fc;"><p>Also note how we just reparsed the file solely for code completion purposes!</p></blockquote>
<p>You mean the preamble? Since code completion is usually invoked after some initial changes, we have to reparse anyway. What you said would only be true if we invoke code completion without changing anything first, which seems rather weird.</p></blockquote>
<p>I mean: We trigger a reparse when we open a document, to get hold of the clang TU to do code completion. If this *does not* create a preamble, then the following call to query code completion items - *after* editing - will create the preamble. The preamble only includes the headers of the file, not the contents of the <tt style="background: #ebebeb; font-size: 13px;">.cpp</tt>.</p></blockquote>
<p>Well, then why don't we just skip the initial reparse, and only reparse after the first edit?</p></blockquote>
<p>Yes, I agree that we should try this out. Note that this will require changes in <tt style="background: #ebebeb; font-size: 13px;">clang/codecompletion/model.cpp</tt> at least, potentially also in other places. I.e. right now this code relies on the fact that we trigger a reparse after opening a document. This then allows us to write code like this there:</p>
<div class="remarkup-code-block" style="margin: 12px 0;" data-code-lang="text" data-sigil="remarkup-code-block"><pre class="remarkup-code" style="font: 11px/15px "Menlo", "Consolas", "Monaco", monospace; padding: 12px; margin: 0; background: rgba(71, 87, 120, 0.08);">auto top = DUChainUtils::standardContextForUrl(m_url);
if (!top) {
qCWarning(KDEV_CLANG) << "No context found for" << m_url;
return;
}
ParseSessionData::Ptr sessionData(ClangIntegration::DUChainUtils::findParseSessionData(top->url(), m_index->translationUnitForUrl(top->url())));
if (!sessionData) {
// TODO: trigger reparse and re-request code completion
qCWarning(KDEV_CLANG) << "No parse session / AST attached to context for url" << m_url;
return;
}</pre></div>
<p>If we only attach the AST after the first edit, it won't be enough. We will also have to trigger a reparse and force attaching the AST/parse session data when code completion is explicitly requested via <tt style="background: #ebebeb; font-size: 13px;">ctrl + space</tt> without editing.</p></blockquote>
<p>How bad would it be if we don't do that and only provide KTextEditor-completions before the first edit? I think it's pretty rare that code completion is immediately requested, typically one starts with a line break, or by removing code, before new code is written.</p>
<blockquote style="border-left: 3px solid #a7b5bf; color: #464c5c; font-style: italic; margin: 4px 0 12px 0; padding: 4px 12px; background-color: #f8f9fc;"><blockquote style="border-left: 3px solid #a7b5bf; color: #464c5c; font-style: italic; margin: 4px 0 12px 0; padding: 4px 12px; background-color: #f8f9fc;"><blockquote style="border-left: 3px solid #a7b5bf; color: #464c5c; font-style: italic; margin: 4px 0 12px 0; padding: 4px 12px; background-color: #f8f9fc;"><p>You usually edit the <tt style="background: #ebebeb; font-size: 13px;">.cpp</tt> file and then you want to get code completion results immediately.</p></blockquote>
<p>We're talking about a one-time delay, all further edits will have a preamble.</p></blockquote>
<p>Yes, but the patch proposed here in its current form just disables the preamble always, no? So I think we agree that this isn't good enough?</p></blockquote>
<p>It shouldn't, unless I'm misunderstanding our usage of <tt style="background: #ebebeb; font-size: 13px;">libclang</tt>. Without <a href="https://clang.llvm.org/doxygen/group__CINDEX__TRANSLATION__UNIT.html#ggab1e4965c1ebe8e41d71e90203a723fe9aa27840716b90a9346019c9f0914f9ab8" class="remarkup-link" target="_blank" rel="noreferrer">the flag</a>, the preamble is created not on the first parse, but on the first reparse of a file. Meaning there is no preamble when I just open a file, but as soon as I edit and the IDE triggers a reparse, the preamble is created and used for subsequent reparsing.</p>
<p>Let's do some math to clarify this. Let</p>
<ul class="remarkup-list">
<li class="remarkup-list-item"><em>t_all</em> be the time needed to parse a source without preamble,</li>
<li class="remarkup-list-item"><em>t_source</em> be the time needed to parse a source with preamble, and</li>
<li class="remarkup-list-item"><em>t_preamble</em> be the time needed to serialize the preamble.</li>
</ul>
<p>Obviously <em>t_source < t_all</em>. Then the times look as follows:</p>
<div class="remarkup-table-wrap"><table style="border-collapse: separate;
border-spacing: 1px;
background: #d3d3d3;
margin: 12px 0;">
<tr><td style="background: #ffffff;
padding: 3px 6px;"></td><td style="background: #ffffff;
padding: 3px 6px;">Time for opening</td><td style="background: #ffffff;
padding: 3px 6px;">Time after first reparse</td><td style="background: #ffffff;
padding: 3px 6px;">Time after <em>n</em>-th reparse</td></tr>
<tr><td style="background: #ffffff;
padding: 3px 6px;">Old</td><td style="background: #ffffff;
padding: 3px 6px;"><em>t_all + t_preamble</em></td><td style="background: #ffffff;
padding: 3px 6px;"><em>t_all + t_source + t_preamble</em></td><td style="background: #ffffff;
padding: 3px 6px;"><em>t_all + n · t_source + t_preamble</em></td></tr>
<tr><td style="background: #ffffff;
padding: 3px 6px;">New</td><td style="background: #ffffff;
padding: 3px 6px;"><em>t_all</em></td><td style="background: #ffffff;
padding: 3px 6px;"><em>2 · t_all + t_preamble</em></td><td style="background: #ffffff;
padding: 3px 6px;"><em>2 · t_all + (n-1) · t_source + t_preamble</em></td></tr>
<tr><td style="background: #ffffff;
padding: 3px 6px;">New - Old</td><td style="background: #ffffff;
padding: 3px 6px;"><em>-t_preamble</em></td><td style="background: #ffffff;
padding: 3px 6px;"><em>t_all - t_source</em></td><td style="background: #ffffff;
padding: 3px 6px;"><em>t_all - t_source</em></td></tr>
<tr></tr>
</table></div>
<p>Now let <em>m_read</em> be the number of documents that are only being read and <em>m_write</em> the number of documents being read and written. Then the cost difference is <em>m_write · (t_all - t_source) - m_read · t_preamble</em>, which is negative iff <em>m_write / m_read < t_preamble / (t_all - t_source)</em>. The right-hand side is a constant that depends on the nature of the project and the hardware one is working on:</p>
<ul class="remarkup-list">
<li class="remarkup-list-item"><em>t_all</em> and <em>t_source</em> include reading from disk or page cache and doing some heavy CPU work, the difference between them is the parsing of the header files,</li>
<li class="remarkup-list-item"><em>t_preamble</em> includes a bit of CPU work and writing to a file (which might be on disk).</li>
</ul>
<p>The left-hand side depends on usage patterns. I use KDevelop a lot to read and browse code without doing any changes, so this would very likely benefit me. But I might be alone in that.</p></div></div><br /><div><strong>REPOSITORY</strong><div><div>R32 KDevelop</div></div></div><br /><div><strong>REVISION DETAIL</strong><div><a href="https://phabricator.kde.org/D18551">https://phabricator.kde.org/D18551</a></div></div><br /><div><strong>To: </strong>aaronpuchert, KDevelop, mwolff, brauch, rjvbb<br /><strong>Cc: </strong>rjvbb, kdevelop-devel, hmitonneau, christiant, glebaccon, domson, antismap, iodelay, alexeymin, geetamc, Pilzschaf, akshaydeo, surgenight, arrowd<br /></div>