Chunk into 400mb pulls

[brookesy2]

I know this is the longest of the long shots, but any possibility in the future of being able to pull from orgs that hit the 400mb metadata API limit by chunking the pulls out?

The error just in case :)

/usr/share/solenopsis/ant/util/sfdc-util.xml:613: Failed to process the request successfully. Cause(LIMIT_EXCEEDED): LIMIT_EXCEEDED: The retrieved zip file exceeded the limit of 419430400 bytes.

[sfloess]

Need ability to denote which Type (maybe dirs) require separate pulls. Should then iterate over them - but be sure to exclude them from the end pull (which will be a pull the rest minus them en-mass).

[sfloess]

Should consider if a property is set, do a chunk pull...otherwise a full pull.

[sfloess]

Consider catching the exception and if caught denote how to do chunking pulls.

[brookesy2]

That would be pretty helpful i think (the catch and then ask) Because potentially you could just pull custom objects or standard objects, which should reduce the amount by a fair bit. I guess its probably not as common to have 400mb of Apex code. Whatever is easiest to do for you though!

[brookesy2]

Was thinking last night, one thing with a prompt is if you want to automate pulls with CI/cron it may mess that up :(

[sfloess]

I don't think prompting is the correct solution more so than emitting an error message on corrective action. And to your point, that wouldn't scale to a CI environment :)

I'll likely code this such that its no harder to chunk objects vs classes. By this I mean, I envision a property that denotes whats chunkable as a space delimited list.

[brookesy2]

That seems like an excellent solution :) Gives a lot of flexibility!

[sfloess]

I'll denote a property with default values of all the types. I'm going to also define two new macrodefs and targets:

• selective-pull
• selective-pull-to-master

If the above are called I'll either use the default value or whatever is specified.

Also, I will iterate over each doing retrieves for each.