Failures when building Komodo Edit from source

[JesperIRL]

The topic is slightly off because I haven’t actually gotten to the part where I build Komodo Edit yet. I get these errors when building Mozilla. But I’m thinking that I’ll continue in this thread when I run into issues in the Komodo build as well I’m building on MacOS Mojave 10.14.6. I’m pretty sure I have installed all dependencies mentioned in BUILD.txt.

 0:04.14 In file included from psutil/_psutil_osx.c:9:
 0:04.14 /opt/local/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7/Python.h:33:10: fatal error: 'stdio.h' file not found
 0:04.14 #include <stdio.h>
 0:04.14          ^~~~~~~~~
 0:04.14 1 error generated.
 0:04.14 error: command '/usr/bin/clang' failed with exit status 1
 0:04.14
 0:04.14 Error processing command. Ignoring because optional. (optional:setup.py:python/psutil:build_ext:--inplace)

It says this one is optional but if I’m going to do this seriously I would like to have all the parts.

 9:37.67 video_capture_qtkit_info.o
 9:37.72 In file included from /Users/jesper/repos/KomodoEdit/mozilla/build/moz3500-ko10.10/mozilla/media/webrtc/trunk/webrtc/modules/v
ideo_capture/mac/qtkit/video_capture_qtkit.mm:11:
 9:37.72 /Users/jesper/repos/KomodoEdit/mozilla/build/moz3500-ko10.10/mozilla/media/webrtc/trunk/webrtc/modules/../../webrtc/modules/video_capture/mac/qtkit/video_capture_qtkit.h:14:9: fatal error: 'QTKit/QTKit.h' file not found
 9:37.72 #import <QTKit/QTKit.h>
 9:37.72         ^~~~~~~~~~~~~~~
 9:37.72 /Users/jesper/repos/KomodoEdit/mozilla/build/moz3500-ko10.10/mozilla/media/webrtc/trunk/webrtc/modules/../../webrtc/modules/video_capture/mac/qtkit/video_capture_qtkit.h:14:9: note: did not find header 'QTKit.h' in framework 'QTKit' (loaded from '/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks')

This one seems to be the actual error that causes the build to fail. The log file contains 1.3 million lines of warnings so I hope that I found the right part here.

This page suggests that QTKit.h is deprecated: https://developer.apple.com/documentation/qtkit

I know this is third party code but I hope that someone might have seen this and know what to do about it.

[careyh]

@JesperIRL

0:04.14 #include <stdio.h>

This looks like the main culprit. Did you make sure you’re using MacOS SDK 10.9?

Ref: GitHub - Komodo/KomodoEdit: Komodo Edit is a fast and free multi-language code editor. Written in JS, Python, C++ and based on the Mozilla platform.

@JesperIRL

It says this one is optional but if I’m going to do this seriously

What says what is optional? I can’t tell if this comment refers to the error above it or below it. If you’re referring to the QTKit error, I’m not aware of anything reason to not include that. We don’t exclude it in our builds so it should be working.

• Carey

[JesperIRL]

Oh my, that’s an old toolchain you got there. I’ll experiment here to see if I can get anything to work. Do you have anyone building on Mac on a regular basis that can give some hints? (Yes, I’m trying to follow the instructions but the problem with these old XCode versions are that they don’t run on any reasonably new MacOS version.)

It also says to use ccache 3.1 - this version is from 2010. Is it the Mozilla parts that sets these requirements, and if so, is there no newer version of that?

To clarify my earlier comment, it’s the first error I got, the one about stdio.h, that ends with the line “0:04.14 Error processing command. Ignoring because optional. (optional:setup.py:python/psutil:build_ext:--inplace)”. That made me think that the first error was not the main problem.

[careyh]

@JesperIRL, for the xcode dependencies, you can install the latest Xcode, then follow the instructions in the README to copy over the necessary SDKs from older version. You don’t need to be running Xcode 5 or 6, just pull pieces out of them.

@JesperIRL

It also says to use ccache 3.1 - this version is from 2010. Is it the Mozilla parts that sets these requirements, and if so, is there no newer version of that?

There are newer versions but for a few reason, one being it’s not a trivial process to upgrade (to put it lightly), we haven’t. Another main reason is that key technology was removed from Mozilla that Komodo relies a huge amount on, that tech being XUL Runner which is the XPCOM component and over all application runner.

Sorry for the delayed response btw.

• Carey

[JesperIRL]

Ok. As far as I can tell I have followed the instructions to the letter. I’m now using the latest XCode with 10.9 and 10.10 SDKs, and MinimumSDKVersion set to 10.9. I got a bit further but am now stuck at:

25:34.76 Undefined symbols for architecture x86_64:
25:34.76   "_CGFontGetGlyphPath", referenced from:
25:34.76       mozilla::gfx::ScaledFontMac::GetPathForGlyphs(mozilla::gfx::GlyphBuffer const&, mozilla::gfx::DrawTarget const*) in Scal
edFontMac.o
25:34.76       mozilla::gfx::ScaledFontMac::CopyGlyphsToBuilder(mozilla::gfx::GlyphBuffer const&, mozilla::gfx::PathBuilder*, mozilla::
gfx::BackendType, mozilla::gfx::Matrix const*) in ScaledFontMac.o
25:34.76 ld: symbol(s) not found for architecture x86_64
25:34.76 clang: error: linker command failed with exit code 1 (use -v to see invocation)
25:34.76 make[5]: *** [XUL] Error 1
25:34.76 make[4]: *** [toolkit/library/target] Error 2
25:34.76 make[4]: *** Waiting for unfinished jobs....
25:34.97 40 warnings generated.
25:34.98 js
25:35.51 ld: warning: could not create compact unwind for _ffi_call_unix64: does not use RBP or RSP based frame
25:35.52 make[3]: *** [compile] Error 2
25:35.52 make[2]: *** [default] Error 2
25:35.52 make[1]: *** [realbuild] Error 2
25:35.52 make: *** [build] Error 2
25:35.52 11512 compiler warnings present.
25:36.05 Notification center failed: Install terminal-notifier to get a notification when the build finishes.
Traceback (most recent call last):
  File "build.py", line 2965, in <module>
    sys.exit( main(sys.argv) )
  File "build.py", line 2961, in main
    return build(args)
  File "build.py", line 2785, in build
    newArgv = targetFunc(argv)
  File "build.py", line 2472, in target_all
    target_mozilla()
  File "build.py", line 2377, in target_mozilla
    buildDir, log.info)
  File "build.py", line 250, in _run_in_dir
    _run(cmd, logstream=None)
  File "build.py", line 232, in _run
    raise OSError("error running '%s': %r" % (cmd, status))
OSError: error running 'python mach --log-file /Users/jesper/repos/KomodoEdit/mozilla/build/moz3500-ko10.10/mozilla/mach.log build ': 2

_CGFontGetGlyphPath has been deprecated and removed in MacOS.

[careyh]

@JesperIRL

_CGFontGetGlyphPath has been deprecated and removed in MacOS.

Sounds like it’s building with the wrong SDK if it’s complaining about deprecated functions.

Double check that you set MinimumSDKVersion to 10.9 in Xcode.app/Contents/Developer/Platforms/macOSX.platform/Info.plist.

Also make sure, if you have multiple Xcodes installed, that you’ve “selected” the write one to be used by running xcode-select -s /Path/To/Xcode.app/.

• Carey

[JesperIRL]

I have verified (again) that MinimumSDKVersion is 10.9, and I only have one XCode installed that is pointed to by xcode-select.

Since the problem is in the linker I would suspect that it’s not the SDK that’s the problem (the function declaration is probably there in the .h files), but rather that the actual implementation is gone in the OS.

[JesperIRL]

My bad, the implementation seems to be there in the SDK. I thought these was just header files but they are actual libraries with all the code.

So back to not having a clue then.

[careyh]

Thanks for confirm. It would be very weird for an SDK to ship with only header files and no implementation.

Can you share [repo-dir]/mozilla/config.py. Maybe that’ll tell us something.

• Carey

[JesperIRL]

It would be very weird for an SDK to ship with only header files and no implementation.

Yes, I was just being stupid. Sorry!

To be sure no other SDK was interfering I removed the other ones and tried with only SDK 10.9 and 10.10 in there. Same result.

config.py:

buildDir = '/Users/jesper/repos/KomodoEdit/mozilla/build'
buildOpt = []
buildTag = None
buildType = 'release'
changenum = 0
compiler = None
configureOptions = ['-k', '10.10', '--options=disable-webrtc']
enableMar = True
gcc = '/usr/bin/clang'
gxx = '/usr/bin/clang++'
jsStandalone = False
komodoCommaVersion = '10,10'
komodoUndottedVersion = '1010'
komodoVersion = '10.10'
mozBuildExtensions = []
mozObjDir = 'ko-rel'
mozSrcHgRepo = '3500'
mozSrcHgTag = None
mozSrcName = 'moz3500'
mozSrcScheme = '3500'
mozSrcType = 'hg'
mozVer = 35.0
mozconfig = '# Options for \'configure\' (same as command-line options).\nmk_add_options MOZ_OBJDIR=@TOPSRCDIR@/ko-rel\nexport CC="/usr/bin/clang -arch x86_64"\nexport CXX="/usr/bin/clang++ -arch x86_64"\nmk_add_options AUTOCONF=autoconf213\nac_add_options --disable-webrtc\nac_add_options --enable-macos-target=10.9\nac_add_options --disable-jemalloc\nac_add_options --enable-help-viewer\nac_add_options --disable-tests\nac_add_options --enable-optimize\nac_add_options --disable-debug\nac_add_options --enable-application=komodo\n'
patchesDirs = ['patches-new']
platform = 'macosx-x86'
platinfo = {'os_name': 'Mac OS X', 'darwin_ver': '18.7.0', 'name': 'macosx-x86', 'os': 'macosx', 'arch': 'x86', 'os_ver': '10.14.6', 'os_build': '18G87'}
pyVer = '2.7'
python = '/Users/jesper/repos/KomodoEdit/mozilla/prebuilt/python2.7/macosx/Python.framework/Versions/2.7/bin/python'
pythonVersion = '2.7'
srcTreeName = 'moz3500-ko10.10'
stripBuild = False
universal = False
withCrashReportSymbols = False
withPGOCollection = False
withPGOGeneration = False

[careyh]

Hey @JesperIRL,

I think I’m stump without knowing more about your computer. Everything looks right to me. I can’t find anything on the internet that’s applicable to this. The only difference is my dev machine is High Sierra because OSX so consistently breaks things with their updates that I’ve just refused to update for some time now. Unfortunately I won’t be changing that practice for this

Another user had issues on Catalina but they were able to build Mozilla no problem. They just had issue RUNNING Komodo:

MacOS Catalina Build Issues

@careyh, It’s not pretty, but I was able to at least get it running. Here are some branches with changes. I’m pretty sure you won’t want to merge the second two, but they might help if you do upgrade. Fix Symlink issue with Catalina Allow for newer ccache version Handle svnversion not available Other changes Change watchdog_fsevents.c from #include <Python/Python.h> to #include <Python.h> Make sure contrib/Conscript has -mmacosx-version-min=10.9 I have random notes on https:/…

Last two suggestions I can think of are to install terminal-notifier as that’s showing up in the error you got and perhaps that’s covering the REAL error some how.

And you can try adding these options to your config command:
--options=disable-elf-hack,disable-gio,enable-gnomeui

I don’t think that last gnome one has any effect but I’m looking at the legacy master build slave and it has it so shrug. My dev machine only has --options=disable-webrtc like yours does.

• Carey

[ssigwart]

It doesn’t look like exactly the same issue I had, but I do have some notes on https://github.com/ssigwart/KomodoEdit/wiki#building that might help. They might not be the right way to do it, but I was able to get it running. I think the MACOSX_DEPLOYMENT_TARGET part and installing /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk/ and /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/ are key parts.

[careyh]

@ssigwart, ya not the same unfortunately.

FYI, MacOSX10.8 shouldn’t be necessary. It was added when we released Code Intel three in IDE. It was not necessary before that and still isn’t in Edit. To be safe, I’m confirming by running a build right now.

I can’t find the note about MACOSX_DEPLOYMENT_TARGET. Is that in one of your PRs @ssigwart?

• Carey

[ssigwart]

No, it’s in my wiki: https://github.com/ssigwart/KomodoEdit/wiki#initial-build

I found it somewhere in the Komodo documentation.

[careyh]

Confirmed 10.8 is not necessary in Edit builds.

[JesperIRL]

@careyh Ok. Some progress thanks to @ssigwart’s wiki. I believe that Mozilla is built now. Adding export MACOSX_DEPLOYMENT_TARGET="10.9" and --options='with-macos-sdk=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk' did the trick.

Now for KomodoEdit. I ran in to this when building:

do "Construct" failed, '.' is no longer in @INC; did you mean do "./Construct"? at /Users/jesper/repos/KomodoEdit/bin/cons.pl line 545.
Building 'bk build quick' cache from installed copy.
Cache created - 0.43
Build time - 1.94

I applied the “manual fixes” from the wiki but got the same result.

[careyh]

@JesperIRL,

I recognize that. I thought I had updated the README for this but apparently not. If you install Perl 5.22 it should solve this problem. I THINK it was Perl 5.26 that changed and caused this but my notes from other work show I used 5.22 in another situation.

I updated the README. Sorry about that one @JesperIRL!

• Carey