Add conversions from vm list operations to emitc #5350

simon-camp · 2021-04-08T12:14:13Z

This adds basic support for all list operations except for the ones dealing with refs.

Things missing include lists as function arguments/results and basic block arguments.

Co-authored-by: Marius Brehler [email protected]

iree/compiler/Dialect/VM/Conversion/VMToEmitC/ConvertVMToEmitC.cpp

simon-camp · 2021-04-09T07:40:59Z

Are we expected to wrap the lists in refs like how it's done in the bytecode dispatcher, or should this be seen as an "implementation detail" of the interpreter?

Current state of the PR (+ deallocation with list API)

iree_vm_type_def_t element_type = iree_vm_type_def_make_value_type(IREE_VM_VALUE_TYPE_I32);
iree_vm_type_def_t* element_type_ptr = &element_type;
iree_vm_list_t* list = NULL;
iree_vm_list_t** list_ptr = &list;
iree_vm_list_create(&element_type, {initial_capacity}, state->allocator, &list);
...
int32_t size = iree_vm_list_size(list);
...
iree_vm_list_release(list); // That's missing currently

Dispatcher (as far as I understand it)

iree_vm_type_def_t element_type = iree_vm_type_def_make_value_type(IREE_VM_VALUE_TYPE_I32);
iree_vm_type_def_t* element_type_ptr = &element_type;
iree_vm_ref_t list_ref = {0};
iree_vm_list_t* list = NULL;
iree_vm_list_t** list_ptr = &list;
iree_vm_list_create(&element_type, {initial_capacity}, state->allocator, &list);
iree_vm_ref_wrap_assign(list, iree_vm_list_type_id(), &list_ref));
...
iree_vm_list_t* list2 = iree_vm_list_deref(*list_ref);
int32_t size = iree_vm_list_size(list2);
...
iree_vm_ref_release(&list_ref);

benvanik · 2021-04-09T14:35:43Z

You don't have to wrap it in a ref but you'll really want to :)
Ideally you should not have to do any special handling for lifetime of list and instead be treating any ref/list/future dict type/etc as the same thing. There's some optimization opportunity where we could use stack-based lists or something but those are ages away. Stuff gets way more complex if you try to specialize, as then things like loading a list from within another list, passing lists to functions, storing lists in variables, or a heterogenous list with multiple types inside of it - some of which are lists and others which are other ref types - get really difficult to handle. Stella is adding some such constructs right now for the core tensorflow/xla import pipelines so you'll want to be building for that generality :)

iree/vm/test/list_ops.mlir

benvanik · 2021-04-13T16:48:29Z

iree/compiler/Dialect/VM/Conversion/VMToEmitC/ConvertVMToEmitC.cpp

+
+  // TODO(simon-camp): Cache the liveness analysis.
+  ValueLiveness liveness;
+  if (failed(liveness.recalculate(funcOp))) {


TODO is fine - note that ValueLiveness is pretty much the slowest possible piece of code ever so FYI you'll hit slowdowns on anything bigger than a unit test.

There's also mlir/Analysis/Liveness.h upstream now which does most of the same stuff (I've been meaning to switch to it) - you may be able to use that here and at least not have to worry about my junk implementation :)

iree/compiler/Dialect/VM/Conversion/VMToEmitC/ConvertVMToEmitC.cpp

benvanik · 2021-04-13T16:51:55Z

iree/compiler/Dialect/VM/Conversion/VMToEmitC/ConvertVMToEmitC.cpp

+        /*result=*/emitc::OpaqueType::get(ctx, "iree_vm_list_t**"),
+        /*operand=*/listOp.getResult());
+
+    rewriter.create<emitc::CallOp>(


These calls can fail - anything with iree_status_t you'll want to insert an IREE_RETURN_IF_ERROR around and propagate the error back up. This is especially important with calls to external modules as for example the HAL failing to load an executable at runtime may fail and if you ignore the error things will get crashy fast.

I've added these, or rather something similar as we can't emit a macro invocation with a function call as argument. So the newly added test at least fails as expected.

We are leaking memory though as we don't release the refs before returning. Let's discuss directly how to properly support this.

iree/compiler/Dialect/VM/Conversion/VMToEmitC/ConvertVMToEmitC.cpp

benvanik · 2021-04-15T15:39:44Z

I think this is good now - just let me know when you're done tweaking and I can merge it!

simon-camp · 2021-04-20T11:24:10Z

@benvanik I've implemented the handling of ref types on top of the register allocation as discussed last week.

However, it seems it is over estimating the number of registers needed. (Or its something about scratch registers, haven't looked into the code to deeply)

I've created some files here.

benvanik

minor fix for an off-by-one on local_refs and moving the value extract to value.h, but otherwise LGTM! I can submit after those fixes

benvanik · 2021-04-20T18:23:38Z

iree/vm/ops.h

+#define VM_GET_REF_ADDRESS(index) &local_refs[index]
+
+#define VM_RELEASE_REFS()                                \
+  for (int i = 0; i < IREE_ARRAYSIZE(local_refs); i++) { \
+    iree_vm_ref_release(VM_GET_REF_ADDRESS(i));          \
+  }


it'd be nice for readability if these took local_refs as an argument - that would make it clearer when looking at the code where these are getting their state from

benvanik · 2021-04-20T18:29:57Z

iree/vm/ops.h

+static inline int64_t vm_list_value_extract_i64(iree_vm_value_t* value) {
+  return value->i64;
+}


we can make this a iree_vm_value_t method in value.h as an iree_vm_value_get_i32 as it's not list specific and can be used with other types as well (future dict/etc). It'd fit nicely alongside iree_vm_value_make_i32 (and i64/etc) and be useful in general.

We could in the future make it assert that the type is as expected and also add an iree_vm_value_cast_i32 that would convert if needed (i64->i32, etc). Some of the list code does these casts already but we may be able to remove a lot of that code if we had these helpers.

benvanik · 2021-04-20T18:56:17Z

iree/compiler/Dialect/VM/Target/C/CModuleTarget.cpp

+    auto ref_initializers =
+        SmallVector<StringRef, 4>{static_cast<size_t>(numRefs), "{0}"};


the max register ordinal is the inclusive max, so you need to +1 here to get the total count

benvanik · 2021-04-20T18:56:50Z

iree/vm/test/list_ops.mlir

+    %c1 = vm.const.i32 1 : i32
+    %list = vm.list.alloc %c1 : (i32) -> !vm.list<i32>
+    vm.list.set.i32 %list, %c0, %c1 : (!vm.list<i32>, i32, i32)
+    vm.return


these are awesome tests - thanks for adding :)

benvanik · 2021-04-20T18:59:52Z

iree/compiler/Dialect/VM/Conversion/VMToEmitC/ConvertVMToEmitC.cpp

+    // TODO(simon-camp): This is expensive as we recalculate the
+    // RegisterAllocation for every alloc in a function. We could make it
+    // compatible with the analysis framework in MLIR which would cache it
+    // automatically IIUC. See here for reference
+    // https://mlir.llvm.org/docs/PassManagement/#analysis-management


nice todo - I think the register allocator predates the analysis stuff actually working/being able to be run from within a conversion - since you are running it outside on the FuncOp (which AFAIK will always be processed first before any nested regions) you'll get this for free when supported!

benvanik · 2021-04-20T19:05:04Z

iree/compiler/Dialect/VM/Conversion/VMToEmitC/ConvertVMToEmitC.cpp

+        /*templateArgs=*/ArrayAttr{},
+        /*operands=*/ArrayRef<Value>{setOp.list()});
+
+    auto listDerefOp = rewriter.create<emitc::CallOp>(


this is a good example of where emitc->emitc canonicalizations can be useful - all these derefs and such can CSE/fold away in a lot of cases before emitting code, which saves the C compiler processing the code from having to perform alias analysis and a bunch of other gnarly things that are unlikely to work right:

In the gist you posted for example v12/v13 and v15/v16 could fold away, as could the checks on them.

v12 = *(v9); v13 = iree_vm_list_deref(v12); VM_RETURN_IF_LIST_NULL(v13); v14 = iree_vm_list_reserve(v13, v2); VM_RETURN_IF_ERROR(v14); v15 = *(v9); v16 = iree_vm_list_deref(v15); VM_RETURN_IF_LIST_NULL(v16); v17 = iree_vm_list_size(v16);

->

v12 = *(v9); v13 = iree_vm_list_deref(v12); VM_RETURN_IF_LIST_NULL(v13); v14 = iree_vm_list_reserve(v13, v2); VM_RETURN_IF_ERROR(v14); v17 = iree_vm_list_size(v13);

Which then can go even further for local values:

v7 = &v6; v8 = iree_vm_list_create(v5, v1, state->allocator, v7); VM_RETURN_IF_ERROR(v8); v9 = VM_GET_REF_ADDRESS(0); v10 = iree_vm_list_type_id(); v11 = iree_vm_ref_wrap_assign(v6, v10, v9); VM_RETURN_IF_ERROR(v11); v12 = *(v9); v13 = iree_vm_list_deref(v12); VM_RETURN_IF_LIST_NULL(v13); v14 = iree_vm_list_reserve(v13, v2);

->

v7 = &v6; v8 = iree_vm_list_create(v5, v1, state->allocator, v7); VM_RETURN_IF_ERROR(v8); v14 = iree_vm_list_reserve(v7, v2);

that's where the power of emitc will really shine - relying on the downstream compiler to do that same transformation with a bunch of those interleaved function calls to external code is never going to be as good as what you can do here :)

(<3 emitc :)

We reuse the register allocation pass and emit an array of refs instead of one ref per value, similar to the bytecode module.

google-cla bot added the cla: yes label Apr 8, 2021

simon-camp commented Apr 8, 2021

View reviewed changes

iree/compiler/Dialect/VM/Conversion/VMToEmitC/ConvertVMToEmitC.cpp Outdated Show resolved Hide resolved

simon-camp force-pushed the emitc-list branch from 93732e6 to 1a96361 Compare April 8, 2021 12:23

marbre requested a review from benvanik April 8, 2021 12:32

marbre added the kokoro:run label Apr 8, 2021

iree-kokoro-bot removed the kokoro:run label Apr 8, 2021

simon-camp force-pushed the emitc-list branch from 1a96361 to ae05781 Compare April 8, 2021 12:37

simon-camp commented Apr 8, 2021

View reviewed changes

iree/compiler/Dialect/VM/Conversion/VMToEmitC/ConvertVMToEmitC.cpp Outdated Show resolved Hide resolved

benvanik approved these changes Apr 8, 2021

View reviewed changes

iree/compiler/Dialect/VM/Conversion/VMToEmitC/ConvertVMToEmitC.cpp Outdated Show resolved Hide resolved

marbre added the kokoro:run label Apr 10, 2021

iree-kokoro-bot removed the kokoro:run label Apr 10, 2021

simon-camp marked this pull request as draft April 12, 2021 13:35

simon-camp force-pushed the emitc-list branch from 33dddd8 to 8889e30 Compare April 13, 2021 11:14

simon-camp marked this pull request as ready for review April 13, 2021 11:31

simon-camp force-pushed the emitc-list branch from 6da8c99 to 73c6ca1 Compare April 13, 2021 12:39

marbre added the kokoro:run label Apr 13, 2021

iree-kokoro-bot removed the kokoro:run label Apr 13, 2021

benvanik requested changes Apr 13, 2021

View reviewed changes

benvanik approved these changes Apr 15, 2021

View reviewed changes

simon-camp force-pushed the emitc-list branch 4 times, most recently from 4af6985 to 568a725 Compare April 20, 2021 10:40

simon-camp requested a review from benvanik April 20, 2021 11:34

marbre added the kokoro:run label Apr 20, 2021

iree-kokoro-bot removed the kokoro:run label Apr 20, 2021

benvanik approved these changes Apr 20, 2021

View reviewed changes

simon-camp added 13 commits April 21, 2021 10:43

WIP list conversion

50f2d43

Implement conversion for list ops

ebe08a7

Replace nullptr with NULL in generated code

a2078e2

Wrap lists in iree_vm_ref_t

c3ff423

Release refs after the last use

1d176d0

Reformat file

2e321ec

Address comments

6b99c31

Fix memory leak

4726fef

We reuse the register allocation pass and emit an array of refs instead of one ref per value, similar to the bytecode module.

Add tests for out of bounds access

a09a0b3

Rephrase comment

00af489

Refactor macros to take an array argument

1f4c359

Fix off by 1 error

8894bdb

Move functions to value.h

2922d85

simon-camp force-pushed the emitc-list branch from e329f40 to 2922d85 Compare April 21, 2021 10:53

simon-camp mentioned this pull request Apr 21, 2021

Add type checks to iree_vm_value_t getters #5542

Closed

Reference tracking issue

a09b1cb

benvanik merged commit 5eae15e into iree-org:main Apr 21, 2021

GMNGeoffrey mentioned this pull request Apr 21, 2021

Merge main -> google #5551

Merged

simon-camp deleted the emitc-list branch April 22, 2021 10:04

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add conversions from vm list operations to emitc #5350

Add conversions from vm list operations to emitc #5350

simon-camp commented Apr 8, 2021 •

edited

Loading

simon-camp commented Apr 9, 2021 •

edited

Loading

benvanik commented Apr 9, 2021 •

edited

Loading

benvanik Apr 13, 2021

benvanik Apr 13, 2021

simon-camp Apr 14, 2021

benvanik commented Apr 15, 2021

simon-camp commented Apr 20, 2021 •

edited

Loading

benvanik left a comment

benvanik Apr 20, 2021

benvanik Apr 20, 2021

benvanik Apr 20, 2021

benvanik Apr 20, 2021

benvanik Apr 20, 2021 •

edited

Loading

benvanik Apr 20, 2021

		auto ref_initializers =
		SmallVector<StringRef, 4>{static_cast<size_t>(numRefs), "{0}"};

Add conversions from vm list operations to emitc #5350

Add conversions from vm list operations to emitc #5350

Conversation

simon-camp commented Apr 8, 2021 • edited Loading

simon-camp commented Apr 9, 2021 • edited Loading

benvanik commented Apr 9, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

benvanik commented Apr 15, 2021

simon-camp commented Apr 20, 2021 • edited Loading

benvanik left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

benvanik Apr 20, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

simon-camp commented Apr 8, 2021 •

edited

Loading

simon-camp commented Apr 9, 2021 •

edited

Loading

benvanik commented Apr 9, 2021 •

edited

Loading

simon-camp commented Apr 20, 2021 •

edited

Loading

benvanik Apr 20, 2021 •

edited

Loading