Fixed timestamp fields comparison

[chrisr3d]

I just discovered this kind of behavior can happen when 2 timestamp values don't have the same time precision:

I did not find in the standard any rule to forbid this difference in time precision, so I guess it should be valid.

The issue comes from the python comparison operators used to compare strings, which will always consider a timestamp value with fewer digits later than the same value with more digits.

I just added an additional test to compare the datetime formatted values before raising a validation exception due to this particular behavior of the strings comparison

In [1]: created = '2022-08-03T07:11:07.029Z'

In [2]: modified = '2022-08-03T07:11:07.029036Z'

In [3]: modified >= created
Out[3]: False

In [4]: from datetime import datetime

In [5]: regex = '%Y-%m-%dT%H:%M:%S.%fZ'

In [6]: datetime.strptime(modified, regex) >= datetime.strptime(created, regex)
Out[6]: True

[ejratl]

I'm struggling with this one a bit. Since I wasn't a maintainer when the pull request was made, I cannot force it to re-run the tests to see if the workflow failures were due an intermittent failure. Locally the tests are also failing, some for unrelated reasons, but it looks like this commit may be impacting the sightings test cases in stix2validator/test/v21/sighting_tests.py
It would also be nice to have a test case added with this change in functionality.

[ejratl]

@chrisr3d Can you make another commit to this pull request? Perhaps update it to the latest upstream code, please. The tests failed and since the PR was made before I became maintainer, I am not allowed to request that they be re-run. But if you add another commit, then I can re-run the failed tests and get this PR added.

[chrisr3d]

I see my changes are causing an issue with the tests on invalid timestamps.
I am investigating on the issue and will push more changes soon.

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 94.23077% with 3 lines in your changes are missing coverage. Please review.

Project coverage is 94.49%. Comparing base (1a7e167) to head (cdd96fa).
Report is 4 commits behind head on master.

Files	Patch %	Lines
stix2validator/v21/musts.py	89.65%	3 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #198      +/-   ##
==========================================
+ Coverage   94.45%   94.49%   +0.03%     
==========================================
  Files          66       66              
  Lines        6173     6208      +35     
==========================================
+ Hits         5831     5866      +35     
  Misses        342      342              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[adulau]

@ejratl This makes sense for me as it was more the issue of precision in the timestamp which now should be fixed. Is it ok for you if I merge it?

[ejratl]

@adulau Yes, please go ahead and do merge it. I was waiting for it to pass the tests and it has done so.

[clenk]

Any chance you could add a unit test or two for this?

[chrisr3d]

I added some checks in the v2.0 validator for some timestamp properties that are also tested in v2.1 with the TIMESTAMP_COMPARE mapping.

And I added unit tests for different STIX objects

• Some were missing any timestamp property test at all
• For those already having tests on the timestamp format, I added test on the timestamp fields comparison

[adulau]

Thank you!