Mapping of Dicts: How to interpret error message?

[stefan-m-lenz]

I have the following code snippet, that works in Julia 0.4:

d = Dict(1 => 2, 3 => 4)
map(kv -> (kv[1], kv[2]^2), d)

In Julia 0.5.0, this gives me the following error message, which I don't understand:

ERROR: ArgumentError: for Associatives, similar requires an element type of Pair;
  if calling map, consider a comprehension instead
 in _collect(::Dict{Int64,Int64}, ::Base.Generator{Dict{Int64,Int64},##65#66}, ::Base.EltypeUnknown,  ::Base.HasLength) at .\array.jl:321
 in map(::Function, ::Dict{Int64,Int64}) at .\abstractarray.jl:1683

Why did this cease to work? How can I understand the error message?

The following works in 0.4 and 0.5 and prints two times "Pair{Int64,Int64}":

for kv in d
    println(typeof(kv))
end

If I can still iterate over the the dictionary and get key-value pairs, so why can't I map them?

[nalimilan]

Here is a short syntax that works:

map(kv -> kv[1]=>kv[2]^2, d)

I would think that we should also accept tuples, though.

[ararslan]

ERROR: ArgumentError: for Associatives, similar requires an element type of Pair;

The Dict type is a subtype of Associative. You can think of Associatives as collections of Pairs. Indeed, if you do collect(d), you'll get an array of pairs. However, note that a tuple is not a pair; Pair is its own type. When you do (a, b), you're constructing a tuple; the notation for constructing pairs is a => b, as Milan showed in his comment. (Note the resemblance to specifying dict entries!)

Looking through the source, calling map on an associative collection calls collect_similar on a Generator that applies the function to each element of the input. This changed was introduced in a2ce230, which was not part of Julia 0.4.

Based on my understanding, collect_similar is trying to collect the object into a container that's similar to what you gave it. Note that similar(d) results in a dict, which as we noted has an element type of Pair. But when you apply your function to each element in the dict, the result's element type is Tuple, which isn't what Julia has allocated for the result, so it errors.

The error is admittedly rather cryptic in your particular scenario, since it's not obvious that map on Associatives would be using similar at all, yet the error happens in/near similar and propagates through to map.

Does that sort of make sense? (To anyone reading: Please let me know if anything I've said is false or misleading!)

[timholy]

Spot on. Worth cross-referencing #17968, which hopefully made the error message clearer, but not clear enough. (The old message would have been no method matching similar(::Dict{Int,Int}, ::Type{Int}) without any hint about using Pair.) Ideas for further improvement are welcome.

[ararslan]

@timholy I think the error is perfectly clear coming from similar, I just think it's not so obvious when it comes out of map instead. Not sure how best to address that. Missed the part in the error message that specifically mentions map. 😑

[timholy]

The fact that you missed it might mean it's not clear enough. The tricky part is coming up with something better. Maybe show examples of valid syntax? E.g.,

Example valid syntaxes:
    map(kv[1]=>abs(kv[2]), dict)     # creates another Dict, of key=>abs(value)
    [kv[2] for kv in dict]           # creates an array of values
    [(kv[1], kv[2]) for kv in dict]  # creates an array of (key,value) tuples

or something like that.

[ararslan]

To be honest, I actually think the existing error message is fine if the line break before the bit about map is removed. I think that would make it sufficiently obvious. Otherwise it bears too much resemblance to the in _collect(...) etc. lines.

[stefan-m-lenz]

Thanks for the explanation, @ararslan. I did not know about the difference between Tuple and Pair.

The error is admittedly rather cryptic in your particular scenario, since it's not obvious that map on Associatives would be using similar at all, yet the error happens in/near similar and propagates through to map.

Yes, I originally had the code

Dict(map(kv -> (kv[1], kv[2]^2), d))

I had no clue about the role of similar and simply wanted to map a Dict to a new Dict.
For me, what makes it especially hard to understand, was "similar requires an element type of Pair". The word similar must be clearly recognizable as a function name. (Would it be possible to write out the parameter types? Or simply parentheses?) I think also other users that are not familiar with the implementation of map will not understand what this means, as this is not documented.

Also, I think the part about "if calling map, consider a comprehension instead" is not very helpful (even if it mentions map since

map(kv -> kv[1] => kv[2]^2, d) 

works perfectly fine - without using a comprehension. So this part of the message does not help to understand what the real problem is actually.

Is it possible to mention the element type? I would suggest the following:

To construct an Associative, similar() requires an element type of Pair, but got element type Tuple{Int64,Int64}....

If I would have read "requires Pair, but got Tuple{Int,Int}", I think I would have been able to fix the error immediately after finding out about the difference between Tuple and Pair. I still would not have understood why exactly similar throws this error. But anyway, that may not be so essential for a user that does not want to dig so deep into the internals of the implementation.

[nalimilan]

Printing the element type is definitely a good idea everywhere it doesn't match expectations. Printing function names differently from standard text would also be nice, but it requires a more general strategy to fix it everywhere.

[KristofferC]

Error messager is easier now:

ERROR: map is not defined on dictionaries