User script: replace String with accents (Javascript)

s1617 · November 8, 2016, 10:05am

I have developed user script for Komodo Edit in JavaScript, whose goal is to replace an accented string in an opened text document by another String (wherever it appears in the selected text in the text document). I tried with the following code:

var the_sel = komodo.editor.selText;
the_sel = the_sel.replace(/CAMIÓN/mg, “Vehicle”);
komodo.editor.replaceSel(the_sel);

The problem is that the accented string (“CAMIÓN” in the example) is never matched. It works if I try to replace any other string that is not accented. The JavaScript file is UTF-8 and also the text document has the same encoding.

Any help or suggestion is really appreciated, as I cannot find a solution. Thank you very much!

nathanr · November 8, 2016, 3:47pm

I tried your snippet and it works fine for me.

Created a UTF-8 file, added the following text:

foo CAMIÓN foo

Then I selected the text and invoked your userscript, CAMIÓN was replaced with “Vehicle”.

s1617 · November 8, 2016, 4:06pm

Thanks!! Then it must be something related to some configuration option, but I cannot find what…

Maybe something related to the JavaScript engine used by Komodo Edit? Any ideas are very welcome.

nathanr · November 8, 2016, 4:49pm

Are you certain your file is marked UTF8 and you are following your own reproduction steps properly?

Try console.log(the_sel) to see how it’s formatted?

@mitchell any ideas?

mitchell · November 8, 2016, 4:56pm

Accented text pasted from Komodo into the browser for this forum is often silently converted (if it isn’t already) to UTF-8, so copy-pasting from the original post directly will likely mask any encoding issues on the user’s side. All signs point to an encoding issue on the user side. I wish encoding problems weren’t so difficult

nathanr · November 8, 2016, 4:56pm

Bah. @s1617 could you attach the snippet as a .txt or upload it somewhere?

mitchell · November 8, 2016, 4:57pm

Or better yet, @s1617 try copy-pasting the snippet from your original forum post and see if it works

s1617 · November 8, 2016, 5:11pm

Thank you for all the suggestions. I still don’t know what the problem is…

I copied and pasted directly your example text (foo CAMIÓN foo) in my text document but still it does not work.

I added the console.log(the_sel) command and this is what I observe in the log file (pystderr.log) after executing the script:

[2016-11-08 18:04:05,426] [WARNING] koScintillaSchemeService: applyScheme:: no indicator for name ‘collab_remote_cursor_4’
[2016-11-08 18:04:05,427] [WARNING] koScintillaSchemeService: applyScheme:: no indicator for name ‘collab_remote_cursor_5’
[2016-11-08 18:04:05,427] [WARNING] koScintillaSchemeService: applyScheme:: no indicator for name ‘collab_remote_cursor_2’
[2016-11-08 18:04:05,427] [WARNING] koScintillaSchemeService: applyScheme:: no indicator for name ‘collab_remote_cursor_3’
[2016-11-08 18:04:05,427] [WARNING] koScintillaSchemeService: applyScheme:: no indicator for name ‘collab_remote_cursor_1’
[2016-11-08 18:04:05,427] [WARNING] koScintillaSchemeService: applyScheme:: no indicator for name ‘collab_remote_change’
[2016-11-08 18:04:05,427] [WARNING] koScintillaSchemeService: applyScheme:: no indicator for name ‘spelling_error’
[2016-11-08 18:04:05,427] [WARNING] koScintillaSchemeService: applyScheme:: no indicator for name ‘collab_local_change’
[2016-11-08 18:04:26,256] [WARNING] console-logger: Key event not available on GTK2: key=“u” modifiers=“control,shift” (1) in chrome://komodo/content/tail/tail.xul:0
[2016-11-08 18:04:26,266] [WARNING] console-logger: Key event not available on some keyboard layouts: key=“v” modifiers=“control,alt” (1) in chrome://komodo/content/tail/tail.xul:0
[2016-11-08 18:04:33,417] [DEBUG] console: log: foo CAMIÓN foo
[2016-11-08 18:04:33,430] [ERROR] console-logger: ReferenceError: view is not defined (2) in chrome://openfiles/content/openfiles.js:585
Traceback (most recent call last):
File “chrome://openfiles/content/openfiles.js”, line 585, in

[2016-11-08 18:06:22,328] [ERROR] console-logger: ReferenceError: view is not defined (2) in chrome://openfiles/content/openfiles.js:585
Traceback (most recent call last):
File “chrome://openfiles/content/openfiles.js”, line 585, in

[2016-11-08 18:06:25,496] [DEBUG] console: log: foo CAMIÓN foo
[2016-11-08 18:06:25,500] [ERROR] console-logger: ReferenceError: view is not defined (2) in chrome://openfiles/content/openfiles.js:585
Traceback (most recent call last):
File “chrome://openfiles/content/openfiles.js”, line 585, in

[2016-11-08 18:06:27,394] [ERROR] console-logger: ReferenceError: view is not defined (2) in chrome://openfiles/content/openfiles.js:585
Traceback (most recent call last):
File “chrome://openfiles/content/openfiles.js”, line 585, in

And here there is the text file that I uploaded in a server (in UTF-8):
https://filetea.me/t1sv0UpiVh9RL2elxTt7pA4Hw

Thanks in advance for any idea.

babobski · November 8, 2016, 7:34pm

Maybe it has to do with you’re default encoding settings:

You can find them under Preferences > international… ( at the top the default encoding settings and a little below the language specific )

babobski · November 8, 2016, 7:44pm

Also you can open the userscrtipt inside komodo using Edit Userscript

In the statusbar you can switch the current file encoding, you probability don’t have UTF-8 there.

s1617 · November 9, 2016, 9:24am

Thank you for all the suggestions. I carefully revised all the encodings and all was perfect, just as indicated in the snapshots.

Through trial and error, I finally solved the problem, although I don’t know the reason: deleting the script file and creating it again, with the same contents, solved the issue. Maybe some incorrect metadata were being attached to the file itself? The strange thing is that Komodo Edit reported all the encodings as expected (UTF-8).

Thank you again for all your help. The important thing is that the problem is solved.

nathanr · November 9, 2016, 4:15pm

That’s very odd. Do you still have the old userscript around by any chance?

s1617 · November 9, 2016, 6:01pm

I noticed the following. The problem seems to be that the encoding in the script file is transformed when re-opening the file.

I see “CAMIÓN” when I am editing the file in Komodo Edit, even after saving the file. However, when I re-open the file, “CAMIÓN” is seen as “CAMIÃ“N”:

the_sel = the_sel.replace(/CAMIÓN/mg, “Vehicle”);
-> changes to
the_sel = the_sel.replace(/CAMIÃ“N/mg, “Vehicle”);

I can try to manually correct the accent, and it looks OK, but again the same problem. If I re-open the file the wrong characters appear again. Komodo’s status bar shows “UTF-8” in all the cases.

I uploaded the problematic script to https://filetea.me/t1sSfC1fAGAQ9Wi3jdb9rusJg (I cannot make it work because the accent is apparently modified when the file is re-opened). The script that works correctly is at https://filetea.me/t1sLSO4jcn3TC6IQaQyML58ew.