Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build emsdk from source #343

Merged
merged 66 commits into from
Jul 12, 2023
Merged
Show file tree
Hide file tree
Changes from 56 commits
Commits
Show all changes
66 commits
Select commit Hold shift + click to select a range
ff79b58
Add dependencies on various Microsoft-built versions of packages
directhex Apr 25, 2023
dffe334
Use standardized build.sh etc, instead of modded repo version
directhex Apr 25, 2023
f9a1c95
Fix SDK inclusion
directhex Apr 25, 2023
eb8392d
Ensure we ignore packages dir
directhex Apr 25, 2023
2e55f2e
Mostly get emsdk recognising msft-built directories
directhex Apr 25, 2023
51d8fc5
Add emscripten to the mix
directhex Apr 26, 2023
f5259d9
fix perms
directhex Apr 26, 2023
215a110
mostly fix build
directhex Apr 26, 2023
820e1eb
More progress
directhex Apr 27, 2023
eb0b8e3
Update latest dependencies
directhex Apr 28, 2023
4eee949
typo
directhex Apr 28, 2023
d76ed89
Update to latest builds
directhex May 3, 2023
343113a
Rename property, to avoid recycling a property used elsewhere
directhex May 3, 2023
c3d5926
Use DestinationFolder, to avoid issues w/ extensionless files
directhex May 3, 2023
184904a
Make Node not filtered out on Linux
directhex May 3, 2023
e2eb755
Make build pass on Linux, if not pack
directhex May 3, 2023
33a5dc1
latest fixes
directhex May 3, 2023
18ddb0d
Allow overriding node path, and ensure it's in a subdirectory
directhex May 3, 2023
45dff62
Fix package name versioning
directhex May 3, 2023
e63f0ed
Delete google closure compiler harder. It keeps coming back!
directhex May 3, 2023
6363f24
Override clang++ binary too
directhex May 3, 2023
38a9f6e
Don't run npm ci, sanitize.py does it for us
directhex May 3, 2023
1da0727
Make sure pack target actually works
directhex May 3, 2023
48e459c
Always pass host tools to embuilder
directhex May 3, 2023
d82437d
fix bad escaping on Python setuptools on OSX
directhex May 4, 2023
4efbb94
Update dependencies
directhex May 9, 2023
ab44bc5
Build correct libraries
directhex May 9, 2023
baf9c47
Delete pyc files, they never match the required Python version anyway
directhex May 9, 2023
305ec05
Merge remote-tracking branch 'origin/main' into msft-built-only
directhex May 9, 2023
fb4f73f
Attempt to make AzDO yml multi-platform
directhex May 9, 2023
bbbbca6
fix indentation
directhex May 9, 2023
5cffb92
Increase timeouts, since the work done is greater & takes longer
directhex May 9, 2023
cd0ec44
Fix temporary artifact folders
directhex May 9, 2023
3eeb824
Maybe make workload upload x64-only
directhex May 9, 2023
dd1b34a
Don't bring in dependencies on manifest-only build
directhex May 9, 2023
fe6c2af
Be more aggressive in skipping build for SB
directhex May 9, 2023
8a63128
fixup! Be more aggressive in skipping build for SB
directhex May 9, 2023
2bacb65
more prebuild elimination
directhex May 9, 2023
d51a6d4
Try again?
directhex May 9, 2023
e80181e
No glob = no use UsagePattern
directhex May 9, 2023
fc0a4ec
Build workload msis on arm64 too
directhex May 9, 2023
bd29ac2
Merge remote-tracking branch 'origin/main' into msft-built-only
directhex Jun 20, 2023
cc320a0
Build against LLVM 16 from Testing
directhex Jun 20, 2023
8b43889
typo
directhex Jun 20, 2023
a5f70fd
clang-16 in wrapper
directhex Jun 20, 2023
3a20589
second half of clang-14->16
directhex Jun 20, 2023
25e26b7
Bump LLVM
directhex Jun 23, 2023
9e01515
Merge remote-tracking branch 'origin/main' into msft-built-only
directhex Jun 23, 2023
70f5e6f
typo
directhex Jun 23, 2023
fece4a0
Bump emscripten package
directhex Jun 24, 2023
81651eb
Bump LLVM
directhex Jul 3, 2023
20292e9
Deal with a distro with `python3` but not `which` (Mariner)
directhex Jul 3, 2023
fe7d8f9
Merge remote-tracking branch 'origin/main' into msft-built-only
directhex Jul 3, 2023
c4d022a
Use itemgroup condition instead of individual conditions
directhex Jul 5, 2023
1536db8
Merge remote-tracking branch 'origin/main' into msft-built-only
directhex Jul 5, 2023
fe87712
Bump Emscripten
directhex Jul 5, 2023
7376b11
Bump Binaryen
directhex Jul 6, 2023
3aa51b9
Merge remote-tracking branch 'directhex/msft-built-only' into msft-bu…
directhex Jul 7, 2023
5830cfe
Only do manifest builds when targeting x64
directhex Jul 7, 2023
00fcaeb
Fix last change
directhex Jul 10, 2023
0180941
Only do workload generation on x64
directhex Jul 10, 2023
cd4dc84
Remove general testing feed
directhex Jul 10, 2023
138240f
Suggestions from Matt RE source-build
directhex Jul 11, 2023
4dca06a
Merge remote-tracking branch 'origin/main' into msft-built-only
directhex Jul 12, 2023
486ee2d
Merge branch 'msft-built-only' of github.com:directhex/emsdk into msf…
directhex Jul 12, 2023
5981326
darc update
directhex Jul 12, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -40,5 +40,7 @@ __pycache__
/spidermonkey
/binaryen

# MSFT ignores
/artifacts
/.dotnet
/.packages
41 changes: 41 additions & 0 deletions Directory.Build.props
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,45 @@
<PropertyGroup Condition="'$(StabilizePackageVersion)' == 'true'">
<StableVersion>$(VersionPrefix)</StableVersion>
</PropertyGroup>

<PropertyGroup>
<_hostArch>$([System.Runtime.InteropServices.RuntimeInformation]::ProcessArchitecture.ToString().ToLowerInvariant)</_hostArch>
<BuildArchitecture>$(_hostArch)</BuildArchitecture>
</PropertyGroup>

<!-- Common properties -->
<PropertyGroup>

<__ProjectDir Condition="'$(__ProjectDir)'==''">$(MSBuildThisFileDirectory)</__ProjectDir>
<ProjectDir>$(__ProjectDir)\</ProjectDir>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

<RootRepoDir>$(ProjectDir)</RootRepoDir>
<ProjectDir Condition="'$(__ProjectDir)'==''">$(MSBuildThisFileDirectory)</ProjectDir>

<BaseIntermediateOutputPath>$(RootRepoDir)artifacts\obj\$(PlatformConfigPathPart)\</BaseIntermediateOutputPath>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just for future reference really: $([MSBuild]::NormalizeDirectory(..)) - this will use the correct slash, and ensure the trailing slash.


<SourceDir>$(__SourceDir)\</SourceDir>
<SourceDir Condition="'$(__SourceDir)'==''">$(ProjectDir)src\</SourceDir>

<RootBinDir>$(__RootBinDir)\</RootBinDir>
<RootBinDir Condition="'$(__RootBinDir)'==''">$(RootRepoDir)artifacts\</RootBinDir>

<BinDir>$(__BinDir)\</BinDir>
<BinDir Condition="'$(__BinDir)'==''">$(RootBinDir)bin\$(PlatformConfigPathPart)\</BinDir>
</PropertyGroup>

<PropertyGroup>
<BinaryenDir>$(ProjectDir)\binaryen\$(VersionPrefix)</BinaryenDir>
<HostBinaryenDir Condition="'$(BuildArchitecture)' == '$(TargetArchitecture)'">$(BinaryenDir)</HostBinaryenDir>
<HostBinaryenDir Condition="'$(BuildArchitecture)' != '$(TargetArchitecture)'">$(BaseIntermediateOutputPath)\host\binaryen\$(BuildArchitecture)</HostBinaryenDir>
<NodeDir>$(ProjectDir)\node\$(VersionPrefix)</NodeDir>
<HostNodeDir Condition="'$(BuildArchitecture)' == '$(TargetArchitecture)'">$(NodeDir)</HostNodeDir>
<HostNodeDir Condition="'$(BuildArchitecture)' != '$(TargetArchitecture)'">$(BaseIntermediateOutputPath)\host\node\$(BuildArchitecture)</HostNodeDir>
<PythonDir>$(ProjectDir)\python\$(VersionPrefix)</PythonDir>
<HostPythonDir Condition="'$(BuildArchitecture)' == '$(TargetArchitecture)'">$(PythonDir)</HostPythonDir>
<HostPythonDir Condition="'$(BuildArchitecture)' != '$(TargetArchitecture)'">$(BaseIntermediateOutputPath)\host\python\$(BuildArchitecture)</HostPythonDir>
<LLVMDir>$(ProjectDir)\llvm\$(VersionPrefix)</LLVMDir>
<HostLLVMDir Condition="'$(BuildArchitecture)' == '$(TargetArchitecture)'">$(LLVMDir)</HostLLVMDir>
<HostLLVMDir Condition="'$(BuildArchitecture)' != '$(TargetArchitecture)'">$(BaseIntermediateOutputPath)\host\llvm\$(BuildArchitecture)</HostLLVMDir>
<EmscriptenDir>$(ProjectDir)\emscripten\$(VersionPrefix)</EmscriptenDir>
</PropertyGroup>
</Project>
2 changes: 2 additions & 0 deletions NuGet.config
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@
<add key="dotnet-tools" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json" />
<add key="dotnet-eng" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json" />
<add key="dotnet8" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet8/nuget/v3/index.json" />
<add key="dotnet8-transport" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet8-transport/nuget/v3/index.json" />
<add key="general-testing" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/general-testing/nuget/v3/index.json" />
directhex marked this conversation as resolved.
Show resolved Hide resolved
</packageSources>
<disabledPackageSources>
<clear />
Expand Down
2 changes: 1 addition & 1 deletion build.cmd
Original file line number Diff line number Diff line change
Expand Up @@ -4,5 +4,5 @@ setlocal
set _args=%*
if "%~1"=="-?" set _args=-help

powershell -ExecutionPolicy ByPass -NoProfile -File "%~dp0eng\common\build.ps1" -restore -build -pack %_args%
powershell -ExecutionPolicy ByPass -NoProfile -File "%~dp0eng\build.ps1" %_args%
exit /b %ERRORLEVEL%
42 changes: 17 additions & 25 deletions build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,15 @@

source="${BASH_SOURCE[0]}"

function is_cygwin_or_mingw()
{
case $(uname -s) in
CYGWIN*) return 0;;
MINGW*) return 0;;
*) return 1;;
esac
}

# resolve $SOURCE until the file is no longer a symlink
while [[ -h $source ]]; do
scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"
Expand All @@ -12,30 +21,13 @@ while [[ -h $source ]]; do
[[ $source != /* ]] && source="$scriptroot/$source"
done

usage()
{
echo "Common settings:"
echo " --help Print help and exit (short: -h)"
echo ""
}

inner_args=
properties=

while [[ $# > 0 ]]; do
opt="$(echo "${1/#--/-}" | awk '{print tolower($0)}')"
case "$opt" in
-help|-h)
usage
exit 0
;;
*)
inner_args="$inner_args $1"
;;
esac
shift
done

scriptroot="$( cd -P "$( dirname "$source" )" && pwd )"

"$scriptroot/eng/common/build.sh" --restore --build --pack $inner_args $properties
if is_cygwin_or_mingw; then
# if bash shell running on Windows (not WSL),
# pass control to powershell build script.
scriptroot=$(cygpath -d "$scriptroot")
powershell -c "$scriptroot\\build.cmd" $@
else
"$scriptroot/eng/build.sh" $@
fi
8 changes: 5 additions & 3 deletions emsdk.py
Original file line number Diff line number Diff line change
Expand Up @@ -1616,9 +1616,9 @@ def installation_path(self):
def installation_dir(self):
dir = self.installation_path()
if path_points_to_directory(dir):
return dir
return os.path.normpath(dir)
else:
return os.path.dirname(dir)
return os.path.dirname(os.path.normpath(dir))

# Returns the configuration item that needs to be added to .emscripten to make
# this Tool active for the current user.
Expand Down Expand Up @@ -1719,7 +1719,9 @@ def is_installed(self, skip_version_check=False):
content_exists = False

if hasattr(self, 'custom_is_installed_script'):
if self.custom_is_installed_script == 'is_optimizer_installed':
if self.custom_is_installed_script == 'lie_and_say_yes':
return True
elif self.custom_is_installed_script == 'is_optimizer_installed':
return is_optimizer_installed(self)
elif self.custom_is_installed_script == 'is_binaryen_installed':
return is_binaryen_installed(self)
Expand Down
Loading