[rkward] /: EXPERIMENTAL Always submit non-user commands UTF8-encoded. Now how can we achieve the same for user commands?

[Thomas Friedrichsmeier]

Git commit 2d6d7f2c95531df17699b51a61f3024e9b8e123a by Thomas Friedrichsmeier.
Committed on 04/07/2016 at 08:33.
Pushed by tfry into branch 'master'.

EXPERIMENTAL Always submit non-user commands UTF8-encoded. Now how can we achieve the same for user commands?

M  +1    -0    ChangeLog
M  +4    -2    rkward/rbackend/rkrbackend.cpp

http://commits.kde.org/rkward/2d6d7f2c95531df17699b51a61f3024e9b8e123a

diff --git a/ChangeLog b/ChangeLog
index 046f07b..6d37e82 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,4 +1,5 @@
 --- Version 0.6.5a - UNRELEASED
+- EXPERIMENTAL: Fix NAs introduced by editing data containing characters that cannot be encoded in the system locale
 - Fixed: Preview boxes followed by a <stretch/> element would not work under some circumstances
 - Fixed: Setting an output CSS file via the GUI did not work
 - Fixed: Bug report feature on Windows did nothing 		TODO: Really test this!
diff --git a/rkward/rbackend/rkrbackend.cpp b/rkward/rbackend/rkrbackend.cpp
index 4c9fdb9..a4f09e2 100644
--- a/rkward/rbackend/rkrbackend.cpp
+++ b/rkward/rbackend/rkrbackend.cpp
@@ -307,6 +307,8 @@ int RReadConsole (const char* prompt, unsigned char* buf, int buflen, int hist)
 					RKRBackend::repl_status.user_command_completely_transmitted = false;
 					RKRBackend::repl_status.user_command_parsed_up_to = 0;
 					RKRBackend::repl_status.user_command_successful_up_to = 0;
+					// TODO FIXME: This is a problem when sending characters which are not encodable in R's current locale. I wish we could simply tell R that the input is UTF8 (as we do in parseCommand()).
+					// Or is there? Alternatively, perhaps we can somehow hack-in UTF8 character positions?
 					RKRBackend::repl_status.user_command_buffer = RKRBackend::this_pointer->current_locale_codec->fromUnicode (command->command);
 					RKTransmitNextUserCommandChunk (buf, buflen);
 					RKRBackend::repl_status.user_command_status = RKRBackend::RKReplStatus::UserCommandTransmitted;
@@ -1133,11 +1135,11 @@ SEXP parseCommand (const QString &command_qstring, RKRBackend::RKWardRError *err
 	SafeParseWrap wrap;
 	wrap.status = PARSE_NULL;
 
-	QByteArray localc = RKRBackend::this_pointer->current_locale_codec->fromUnicode (command_qstring);		// needed so the string below does not go out of scope
+	QByteArray localc = command_qstring.toUtf8 (); // needed so the string below does not go out of scope
 	const char *command = localc.data ();
 
 	PROTECT(wrap.cv=Rf_allocVector(STRSXP, 1));
-	SET_STRING_ELT(wrap.cv, 0, Rf_mkChar(command));
+	SET_STRING_ELT(wrap.cv, 0, Rf_mkCharCE(command, CE_UTF8));
 
 	// Yes, if there is an error in the parse, R does jump back to toplevel!
 	// trying to parse list(""=1) is an example in R 3.1.1