Conditional code preprocessing

[dnfield]

I'm primarily concerned about this for Flutter, where I can't use dart:mirrors.

As a library author, I would like to be able to publish code that conditionally uses new functionality if it's available in a downstream consumer's SDK version.

I could imagine implementing this by checking:

1. Whether a compile time variable is set.
2. Whether a method/property is defined/exists in the Flutter SDK API that my library consumes.

For these examples, assume I want to decode a UTF8 string and use data in that string to draw a circle with a particular type of gradient; in the latest SDK, I can draw exactly that gradient, whereas in older SDKs I want to log a warning and draw something close to the desired gradient (perhaps in a more computationally expensive manner or in a manner that is lower fidelity compared with the expected output now possible in the latest SDK).

For example, using C-style preprocessor notation

#if FLUTTER_SDK >= 0.4.5
// Use new and better methods, use alternatives to deprecated methods, e.g. `utf8.decode`
// or new gradient shading methods that are just getting added.
#else
// Fall back or simply don't provide functionality, or use a method
// that isn't deprecated in older version, e.g. `UTF8.decode`, or using a close match 
// on the gradient or perhaps just skipping the rendering for the unavailable 
// gradient method(s).
#endif

or:

#if IS_DEFINED(sdkClass.newCoolMethod)
// code that uses `sdkClass.newCoolMethod()`
#else
// code that uses some work around, or perhaps does nothing at all
#endif

or, using something like JavaScript:

if (sdkClass.newCoolMethod) { // but no reflection.
  sdkClass.newCoolMethod();
}

Basically, I'd like to avoid needing to maintain multiple versions of my library to support Flutter beta, dev, and/or master channels. I'm OK with users on Beta not getting cutting edge functionality, and I'm even OK with maintaining a build configuration file (or section of pubspec.yaml) to drive this; but I'd like to avoid a very confusing scenario where I have to keep going back and checking if I can migrate functionality to my "beta" tracking package from my "dev" tracking package - just let the compiler include that functionality if it's available in the SDK, and skip it (or use a specified alternative) if it's not.

Although I'm using C-style preprocessing to illustrate here, I'm not looking for a full preprocessor. I'd be very open to other ideas or suggestions that don't involve reflection at runtime.

[robertmuth]

some mechanism like this would also be useful for libraries that
come in two flavors: one depending on 'dart:html' and one that does not.

[a-siva]

/cc @leafpetersen @lrhn

[leafpetersen]

In the Dart 2 time frame, we will have some (but not all of what you are asking for here). You will be able to conditionally import/export code controlled by a fixed set of platform defined flags. We will not have finer granularity (e.g. platform version) in that time frame (and no current plans for adding it later). Follow #32960 for more details.

[dnfield]

So it certainly could be achieved as a preprocessing step before passing to Dart if it's not targetted for the language. My only fear with that is fragmentation (there's the coffeedart preprocessor, and the typedart preprocessor, and the danfielddart preprocessor, etc). That, and the fact that it wouldn't be recognized by the analyzer in any meaningful way.

[lrhn]

I do want a feature like this. It enables gradual migration of new, breaking, language features.

[robertmuth]

BTW: this is the best documentation on conditional imports I could find:

https://medium.com/@dvargahali/dart-2-conditional-imports-update-16147a776aa8

[dnfield]

This is coming up as something that would help Flutter framework implement cross-platform features in ways that are especially challenging today - such as supporting both dart:html and dart:io, or supporting Fuchsia specific things that we don't expect will be available for other platforms.

[dnfield]

/cc @amirh

[robertmuth]

After having used conditional imports on a few occasions I find them to be rather clumsy.
Typically I use them to redirect code to a stub implementation, which usually affects many files.

A much cleaner mechanism in my opinion would allow me to specify this redirection in only place.
For example, I would like to say
"for this build target, redirect all uses of dart:html to stubbed_out_html.dart"

[lrhn]

If you just want to implement an entire API twice, without forwarding from stubs, you can use conditional exports.

library platform_dependent_html;
export "stubbed_out_html.dart"
    if (dart.library.html) "html.dart";

Then you can import this library, and get real HTML if it's there, or stubbed_out_html.dart if not.

We do not provide a way for a build to replace an existing library that someone else has asked for explicitly.

[natebosch]

If you just want to implement an entire API twice, without forwarding from stubs, you can use conditional exports.

I don't think conditional exports are working everywhere currently. #34179

[VladimirCores]

Much needed feature!
And you can find inspiration in Haxe Conditional Compilation it helps us a lot https://haxe.org/manual/lf-condition-compilation.html

[maxim-saplin]

I'm definetly missing C# pre-processor features, different build configurations defining own symbols and condition compilation with #if statements

[navaronbracke]

@lrhn has there been any more thought about this recently? I was thinking about this exact proposal in lieu of having a much cleaner way of doing io/html implementations of let's say a service class (with regards to Flutter).

Currently what I have to do to get it working is the following.
Not sure if there is a cleaner way for my current approach either.
(Bear in mind, this is in a standalone project, specifically not a package)

1. A stub file with a global method

• contains a top-level method like MyService getMyServiceInstance() => throw UnimplementedError('this is a stub');

2. A common interface that defines the service, MyService

• contains a conditional import that imports the correct implementation (or the stub)
• contains a factory constructor of MyService that uses the global method as implementation

3. Implementations for io/web

• override the global stub method and pass in their own instance

With pre-processor features like in C++/C# I can get rid of the stub file, (the factory constructor) and the separate implementations, therefore keeping everything in one class.

Something like this

#ifdef library.dart.io
import 'dart:io';
#elifdef library.dart.html
import 'dart:html';
#endif

class MyService {
  bool someFunction(){
#ifdef library.dart.io
    return true;
#elifdef library.dart.html
    return false;
#else
    throw UnimplementedError();
#endif
  }
}

[fuzzybinary]

I'd like to know if the Dart team is at least considering this, especially for package maintainers, as we can't assume people using our packages will upgrade to the most recent version of Flutter or Dart.

A few things I have encountered recently that could be easily solved using conditional compilation, but require workarounds (or worse, maintaining multiple versions of the same library).

• The addition of two methods to HttpClient in Dart 2.17 (required I ship two versions of my package, one that supports < 2.16 and one >= 2.17).
• The change to make WidgetsBinding.instance non-optional
• Upcoming changes to prefer using PlatformDispatcher.instance.onError over runZonedGuarded

[dnfield]

The more I think about this, the more I think what I'd really like here is API availability capabilities.

IOW, it's not that I really want preprocessor defines in Dart, it's that I want to know if some API is available (e.g. whether dart:ui as a whole is avialable, or whether dart:ui has a function called someNewAwesomeThing, or whether object X has a const constructor not, etc.).

[martin-braun]

• The change to make WidgetsBinding.instance non-optional

I figured out a way to circumvent this issue.

/// This allows a value of type T or T? to be treated as a value of type T?.
///
/// We use this so that APIs that have become non-nullable can still be used
/// with `!` and `?` on the stable branch.
// TODO: Remove this, once Flutter 2 support is dropped
// See https://github.com/flutter/flutter/issues/64830
T? _ambiguate<T>(T? value) => value;

Now you can do _ambiguate(WidgetsBinding.instance)!.addPostFrameCallback(... without getting a warn.

[VladimirCores]

In Haxe language conditional compilation looks like this:

class Main {
  public static function main() {
    #if !debug
    trace("ok");
    #elseif (debug_level > 3)
    trace(3);
    #else
    trace("debug level too low");
    #end
  }
}

https://haxe.org/manual/lf-condition-compilation.html

[ohir]

@dnfield

it's not that I really want preprocessor defines in Dart, it's that I want to know if some API is available (e.g. whether dart:ui as a whole is avialable, or whether dart:ui has a function called someNewAwesomeThing, or whether object X has a const constructor not, etc.).

But by what mechanics would you act on this knowledge? At the runtime all mentioned pieces can be known just by a generated consts – as Flutter installation comes with Dart sources.

[dnfield]

@ohir - there's no sane way currently to check at runtime whether a particular method (or method signature) is defined, and certain things are checked at compile time (like whether a const constructor is available or not).

I'm mainly thinking of how availability macros work in Objective-C on iOS/macOS. There I can do things like have code conditionally work if and only if I'm using a particular SDK version.

So for example, as a package author, I want to use some new exciting API that Flutter offers, but I have no way to check whether that API exists today (ignoring doing things like casting to dynamic and trying to invoke a method which is horrible for many reasons). I also don't have a way to have a single package offer better behavior depending on which version of the Flutter/Dart SDKs are in use. E.g.

  if (DartVersion > xxx) {
   // use cool new feature that is faster
  } else {
    // use slower fallback
  }

You can come close to getting this by having minimum version requirements in your pubspec, but pub has a hard time solving for strange and overlapping constraints - and it ends up requiring package maintainers to maintain multiple versions of their packages that may not even make it to their users correctly because of pub.

[fuzzybinary]

I asked @munificent about this on Twitter and he mentioned that a general case pre-processor likely isn't coming any time soon, because it requires a lot of the tooling support for things like changing targets (think the target / platform selector on Visual Studio), and since we have Platform.isX, kDebugMode, kIsWeb and a few other compile time consts using Platform.environment, the one thing library creators / maintaners are lacking is what @dnfield is describing.

I feel like Dart could copy swift's available attributes and compiler directive for use with Dart and Flutter versions: https://www.avanderlee.com/swift/available-deprecated-renamed/

So similar to what Dan is suggesting we could have code like:

class MyHttpClient : HttpClient {
  @available(Dart >= '2.17')
  @override
  set connectionFactory(
          Future<ConnectionTask<Socket>> Function(
                  Uri url, String? proxyHost, int? proxyPort)?
              f) =>
      innerClient.connectionFactory = f;
}

Or:

if (#available(Flutter > 3)) {
  WidgetsBinding.instance.addObserver(this)
} else {
   WidgetsBinding?.instance.addObserver(this)
}

In both cases, the compiler should skip any code that doesn't meet the available criteria.

Since this mostly affects library maintainers, and we're likely to use fvm to change versions of Flutter, any support needed in tooling other than the analyzer could lag.

I'd be interested to hear what the Dart team thinks...

[ohir]

how availability macros work in Objective-C on iOS/macOS

Similar functionality likely could be provided for Dart/Flutter own inventory, but for a substantial effort. I have no high hopes this will ever happen (ie. maintaining full lifecycles and supporting them thorough the toolchain).

pub has a hard time solving for strange and overlapping constraints

What about library author testing for all possible permutations (of execution path configurations) that are result of the pub solver coming to a given set of dependencies peculiar to the site of use? You can control, and that just to an extent, only your package's direct dependencies (version-wise).

I have no way to check whether that API exists today

I understand that you have no way to check whether that API exists at your package user's installation, am I correct? If I am, I know of no "contemporary automagic solution", but I think that 'ol pale ./configure might work (if contemporary users were used to do eg. a dart test configure step). There, in a first test, could be a place to do that ugly cast-try-catch research for API availability. Done once.

[q25001560]

Members! please add the conditional compilation feature to Dart just like C or Apple's Swift. It's important!

[BradKwon]

I am also missing C# pre-processor directives. Hope Dart will have this feature soon as well.

[esDotDev]

We're missing this over here as well:
gskinner/flutter_animate#41

This is a bit downstream, but it would be quite nice to have the flutter versions eventually stored off as compilation consts:

#if flutter_3_3_1_or_newer
 //
#elif flutter_3_3_0
//
#else
 //
#endif 

So it would be cool if somehow that usecase was enabled (not just dart version).

[martin-braun]

It would be nice to have kDebugMode as pre-processor variable as well. It would also be nice to have this ability in the pubspec.yaml somehow, or fix dev_dependencies already.

dev_dependencies apparently are shipped into the release build of Android (when imported in code, despite being used behind kDebugMode exclusively).

[mraleph]

dev_dependencies apparently are shipped into the release build of Android (when imported in code, despite being used behind kDebugMode exclusively).

If all uses are hidden behind if (kDebugMode) I would expect Dart code to be tree-shaken. Native dependencies will still be pulled in though.

Do you have an example?

[martin-braun]

Native dependencies will still be pulled in though.

Yes, exactly this is my problem. Please see the backgrounds for my complaint here: Baseflow/flutter-geolocator#1296 (comment)

The package in question that causes this problem though is safe_device, see ufukhawk/safe_device#33

I only ever need the package in debug mode, so I assumed defining it in dev_dependencies would strip the native bits in production as well.

Apparently, the package causes a lot of problems in the Android release build that are massive and critical, but I still rely on the package during development, it makes things so much easier for me, because there are a lot of things that can only be tested on a real device.

Just for the record though: I switched to device_info_plus and my issue is solved, but the case was very real. For the sake of lightweight apps, dev_dependencies should be fully excluded from release builds, also the native bits.

Thanks.

[andynewman10]

If all uses are hidden behind if (kDebugMode) I would expect Dart code to be tree-shaken. Native dependencies will still be pulled in though.

Why will native dev_dependencies be pulled? dev_dependencies assets are also shipped in the release build, which seems also wrong to me.

I see two important-to-fix bugs here, as there is no easy way for developers to work on development features and have the possibility to easily deactivate these features for release builds.

[yaminet1024]

I also need this to distinguish between different flutter sdk versions. Is there any update here?

[munificent]

No update that I'm aware of.

[fuzzybinary]

@munificent Is this something macros could be hacked a bit to do?

For example, could you write:

  @dart_minimum('2.17')
  @override
  set connectionFactory(
          Future<ConnectionTask<Socket>> Function(
                  Uri url, String? proxyHost, int? proxyPort)?
              f) =>
      innerClient.connectionFactory = f;

And have the macro read current Dart version and remove the attributed code block for versions under 2.17? Or are macros only additive?

This is not ideal (It'd still love a #available style system) but might be a good temporary workaround.

[lrhn]

Macros are mainly additive, but they can bypass existing code entirely by not calling it.
An augmentation for the setter could be

augment set connectionFactory(Future<ConnectionTask<Socket>> Function(
                  Uri url, String? proxyHost, int? proxyPort)?
              f) {}

or maybe a body of { super.connectionFactory = f; } if there is a super-implementation to fall back on.

The function won't stop existing, but it will stop doing anything.

A bigger questions is whether the macro has access to the language version (but I guess you could just have one version of the macro package per SDK release, say one with an SDK constraint of >= 3.7.0 < 3.8.0 which knows that it's definitely running on SDK 3.7.X and language version 3.7.
(Maybe we should just make the information available, so we don't get people filling up the package version database like that.)

[munificent]

@munificent Is this something macros could be hacked a bit to do?

Not really. By design, macros don't have access to any configuration information. If they did, then the compilers would have to be very careful to run macros separately for each possible configuration and track their outputs separately.

It would definitely be useful of macros could do config-specific stuff, but it has a lot of implications for developer experience, complexity, and compile time performance, so at least for now we are avoiding it by not giving macros access to that information.

cc @jakemac53