fix: Add __eq__, __hash__ to SparkSource for correct comparison

[ElliotNguyen68]

What this PR does / why we need it: When apply on old sparksource, even though the path or table or query changed, but store not able to know that there is the difference between old version in store with current version, based on eq function in spark source is only inherited from datasource, this only compare between very common attribute of a datasource:
self.name != other.name
or self.timestamp_field != other.timestamp_field
or self.created_timestamp_column != other.created_timestamp_column
or self.field_mapping != other.field_mapping
or self.date_partition_column != other.date_partition_column
or self.description != other.description
or self.tags != other.tags
or self.owner != other.owner
so when new change comming, store not find out and will not update the location behind.
Add eq will also need to add hash_ function for sparksource class, which will go directly to parent class datasource

Which issue(s) this PR fixes:

Fixes #

[ElliotNguyen68]

Hi @tokoko hope you review the pr .

[HaoXuAI]

This probably applies to other data source as well. It looks good to me for now

[tokoko]

Sorry for nitpicking :) Won't this look better as a single expression? something like return table == table and query == query ...

[ElliotNguyen68]

Because they have priority, I see that there is a kind of hierarchy in the way we get the data out of sparksource in get_table_query_string function, if not table provided then read query if not read path. So I assume that there is a case that like this
sparksourceA(table = 'tableA', query=None) and after that it change to sparksourceA(table = 'tableA', query='select some thing from sometable'), how should we compare in this case. I still have some concern about the logic of providing table, query and path when init sparksource, isn't it we just need 1 of those to be provided, and 2 remained will be null, should we have some contraint on this? If yes so we can do some things like return table == table and query == query ... @tokoko @HaoXuAI

[tokoko]

Those are good points, but I think we should still simply check for absolute equality, so priority doesn't really matter, if there is a change in query or if user changes query to None and switches to using table, this method should return False. In some cases, the change might not actually have an impact i guess, but it's still a change. So a single expression should be fine.

As for the question about 3 competing parameters (table, query and path), some sort of a constraint is probably a good idea, but imho the whole idea of using SparkSource for everything was a really bad decision in the first place. If the user wants to register a parquet folder as a data source for example, i believe the way to do that should be by providing FileSource object instead of a SparkSource one... To achieve that, we need to teach SparkOfflineStore how to read FileSource and deprecate path parameter from SparkSource... but that's a discussion for another day and a little out of scope here :)

[ElliotNguyen68]

okay I think we can by pass the order, thanks @tokoko

[ElliotNguyen68]

Hi @HaoXuAI , I think we can merge the pr .

[ElliotNguyen68]

Hi @HaoXuAI , can you help me with the pr ?

[HaoXuAI]

done. thanks for the work 👍

[HaoXuAI]

LGTM

[tokoko]

@HaoXuAI you can close the other one as well #3819, thanks