Skip to content

Commit

Permalink
cli: add "op abandon root..head" command that "reparents" operations
Browse files Browse the repository at this point in the history 
In order to implement GC (martinvonz#12), we'll need to somehow prune old operations.
Perhaps the easiest implementation is to just remove unwanted operation files
and put tombstone file instead (like git shallow.) However, the removed
operations might be referenced by another jj process running in parallel. Since
the parallel operation thinks all the historical head commits are reachable, the
removed operations would have to be resurrected (or fix up index data, etc.)
when the op heads get merged.

The idea behind this patch is to split the "op log" GC into two steps:
 1. recreate operations to be retained and make the old history unreachable,
 2. delete unreachable operations if the head was created e.g. 3 days ago.
The latter will be run by "jj util gc". I don't think GC can be implemented
100% safe against lock-less append-only storage, and we'll probably need some
timestamp-based mechanism to not remove objects that might be referenced by
uncommitted operation.

FWIW, another nice thing about this implementation is that the index is
automatically invalidated as the op id changes. The bad thing is that the
"undo" description would contain an old op id. It seems the performance is
pretty okay.

This patch also includes support for "op abandon root.." because it can be
implemented for free. I'm not sure what should be checked as prerequisite.
I made it require "op restore root && op abandon root.." for now.
  • Loading branch information
@yuja
yuja committed Jan 2, 2024
1 parent 29d9786 commit f977926
Show file tree
Hide file tree
Showing 3 changed files with 255 additions and 1 deletion.
3 changes: 3 additions & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
* `jj branch` has gained a new `rename` subcommand that allows changing a branch
name atomically. `jj branch help rename` for details.

* New `jj op abandon` command is added to clean up the operation history. If GC
is implemented, Git refs and commit objects can be compacted.

### Fixed bugs

* Command aliases can now be loaded from repository config relative to the
Expand Down
129 changes: 128 additions & 1 deletion cli/src/commands/operation.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,12 +12,17 @@
// See the License for the specific language governing permissions and
// limitations under the License.

use std::io::Write as _;
use std::slice;

use clap::Subcommand;
use jj_lib::backend::ObjectId;
use jj_lib::op_walk;
use jj_lib::repo::Repo;

use crate::cli_util::{user_error, CommandError, CommandHelper, LogContentFormat};
use crate::cli_util::{
user_error, user_error_with_hint, CommandError, CommandHelper, LogContentFormat,
};
use crate::graphlog::{get_graphlog, Edge};
use crate::operation_templater;
use crate::templater::Template as _;
Expand All @@ -29,6 +34,7 @@ use crate::ui::Ui;
/// https://github.com/martinvonz/jj/blob/main/docs/operation-log.md.
#[derive(Subcommand, Clone, Debug)]
pub enum OperationCommand {
Abandon(OperationAbandonArgs),
Log(OperationLogArgs),
Undo(OperationUndoArgs),
Restore(OperationRestoreArgs),
Expand Down Expand Up @@ -89,6 +95,24 @@ pub struct OperationUndoArgs {
what: Vec<UndoWhatToRestore>,
}

/// Abandon operation history
///
/// To discard old operation history, use `jj op abandon ..<operation ID>`. It
/// will abandon the specified operation and its all ancestors. The descendants
/// will be reparented onto the root operation.
///
/// To discard recent operations, run `jj op restore <operation ID>` first to
/// restore the repository to the desired state. If the repository looks good,
/// use `jj op abandon <operation ID>..` to remove the operations.
///
/// The abandoned operations, commits, and other unreachable objects can be
/// later garbage collected by using `jj util gc` command.
#[derive(clap::Args, Clone, Debug)]
pub struct OperationAbandonArgs {
/// The operation range to abandon
operation: String,
}

#[derive(Copy, Clone, Debug, PartialEq, Eq, PartialOrd, Ord, clap::ValueEnum)]
enum UndoWhatToRestore {
/// The jj repo state and local branches
Expand Down Expand Up @@ -249,12 +273,115 @@ fn cmd_op_restore(
Ok(())
}

fn cmd_op_abandon(
ui: &mut Ui,
command: &CommandHelper,
args: &OperationAbandonArgs,
) -> Result<(), CommandError> {
let mut workspace_command = command.workspace_helper(ui)?;
let repo = workspace_command.repo().clone();
let current_head_op = repo.operation();
let Some((root_op_str, head_op_str)) = args.operation.split_once("..") else {
return Err(user_error_with_hint(
format!("Unsupported expression: {}", args.operation),
"Specify operation range as ..head_id or root_id..",
));
};
let abandon_root_op = if root_op_str.is_empty() {
// TODO: Introduce a virtual root operation and use it instead.
op_walk::walk_ancestors(slice::from_ref(current_head_op))
.last()
.unwrap()?
} else {
workspace_command.resolve_single_op(root_op_str)?
};
let abandon_head_op = if head_op_str.is_empty() {
current_head_op.clone()
} else {
workspace_command.resolve_single_op(head_op_str)?
};

if abandon_head_op == *current_head_op {
// The minimum requirement here is that the working copy tree is
// identical so the operation id can be remapped, but let's force user
// to do "op restore"/"undo" first. There's no "op-heads undo" command
// to recover from bad "op abandon".
let new_view = abandon_root_op.view()?;
let jj_lib::op_store::View {
head_ids: cur_head_ids,
public_head_ids: cur_public_head_ids,
local_branches: cur_local_branches,
tags: cur_tags,
remote_views: cur_remote_views,
git_refs: _,
git_head: _,
wc_commit_ids: cur_wc_commit_ids,
} = repo.view().store_view();
let jj_lib::op_store::View {
head_ids: new_head_ids,
public_head_ids: new_public_head_ids,
local_branches: new_local_branches,
tags: new_tags,
remote_views: new_remote_views,
git_refs: _,
git_head: _,
wc_commit_ids: new_wc_commit_ids,
} = new_view.store_view();
if cur_head_ids != new_head_ids
|| cur_public_head_ids != new_public_head_ids
|| cur_local_branches != new_local_branches
|| cur_tags != new_tags
|| cur_remote_views != new_remote_views
|| cur_wc_commit_ids != new_wc_commit_ids
{
return Err(user_error_with_hint(
"Cannot roll back to unrelated repository state",
"Run `jj op restore` first to move to the desired state",
));
}
// TODO: Maybe the current git_refs should be copied to the new view?
}

// Acquire working copy lock early as we'll need to remap the operation id.
// This also prevents "op abandon --at-op" where @ is not the op heads.
let (locked_ws, _) = workspace_command.start_working_copy_mutation()?;
// Reparent descendants, count the number of abandoned operations.
let stats = op_walk::reparent_range(
repo.op_store().as_ref(),
slice::from_ref(&abandon_head_op),
slice::from_ref(current_head_op),
&abandon_root_op,
)?;
if stats.rewritten_count == 0 {
writeln!(
ui.stderr(),
"Abandoned {} operations.",
stats.unreachable_count,
)?;
} else {
writeln!(
ui.stderr(),
"Abandoned {} operations and reparented {} descendant operations.",
stats.unreachable_count,
stats.rewritten_count,
)?;
}
let [new_head_id] = stats.new_head_ids.try_into().unwrap();
if current_head_op.id() != &new_head_id {
repo.op_heads_store()
.update_op_heads(slice::from_ref(current_head_op.id()), &new_head_id);
}
locked_ws.finish(new_head_id)?;
Ok(())
}

pub fn cmd_operation(
ui: &mut Ui,
command: &CommandHelper,
subcommand: &OperationCommand,
) -> Result<(), CommandError> {
match subcommand {
OperationCommand::Abandon(args) => cmd_op_abandon(ui, command, args),
OperationCommand::Log(args) => cmd_op_log(ui, command, args),
OperationCommand::Restore(args) => cmd_op_restore(ui, command, args),
OperationCommand::Undo(args) => cmd_op_undo(ui, command, args),
Expand Down
124 changes: 124 additions & 0 deletions cli/tests/test_operations.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -279,6 +279,130 @@ fn test_op_log_configurable() {
assert!(stdout.contains("my-username@my-hostname"));
}

#[test]
fn test_op_abandon_ancestors() {
let test_env = TestEnvironment::default();
test_env.jj_cmd_ok(test_env.env_root(), &["init", "repo", "--git"]);
let repo_path = test_env.env_root().join("repo");

test_env.jj_cmd_ok(&repo_path, &["commit", "-m", "commit 1"]);
test_env.jj_cmd_ok(&repo_path, &["commit", "-m", "commit 2"]);
insta::assert_snapshot!(test_env.jj_cmd_success(&repo_path, &["op", "log"]), @r###"
@ bacc8030a969 [email protected] 2001-02-03 04:05:09.000 +07:00 - 2001-02-03 04:05:09.000 +07:00
│ commit a8ac27b29a157ae7dabc0deb524df68823505730
│ args: jj commit -m 'commit 2'
◉ bb26fe31d66f [email protected] 2001-02-03 04:05:08.000 +07:00 - 2001-02-03 04:05:08.000 +07:00
│ commit 230dd059e1b059aefc0da06a2e5a7dbf22362f22
│ args: jj commit -m 'commit 1'
◉ 19b8089fc78b [email protected] 2001-02-03 04:05:07.000 +07:00 - 2001-02-03 04:05:07.000 +07:00
│ add workspace 'default'
◉ f1c462c494be [email protected] 2001-02-03 04:05:07.000 +07:00 - 2001-02-03 04:05:07.000 +07:00
initialize repo
"###);

// Abandon old operations. The working-copy operation id should be updated.
let (_stdout, stderr) = test_env.jj_cmd_ok(&repo_path, &["op", "abandon", "..@-"]);
insta::assert_snapshot!(stderr, @r###"
Abandoned 2 operations and reparented 1 descendant operations.
"###);
insta::assert_snapshot!(test_env.jj_cmd_success(&repo_path, &["debug", "workingcopy"]), @r###"
Current operation: OperationId("fb5252a68411468f5e3cf480a75b8b54d8ca9231406a3d0ddc4dfb31d851839a855aca5615ba4b09018fe45d11a04e1c051817a98de1c1ef5dd75cb6c2c09ba8")
Current tree: Merge(Resolved(TreeId("4b825dc642cb6eb9a060e54bf8d69288fbee4904")))
"###);
insta::assert_snapshot!(test_env.jj_cmd_success(&repo_path, &["op", "log"]), @r###"
@ fb5252a68411 [email protected] 2001-02-03 04:05:09.000 +07:00 - 2001-02-03 04:05:09.000 +07:00
│ commit a8ac27b29a157ae7dabc0deb524df68823505730
│ args: jj commit -m 'commit 2'
◉ f1c462c494be [email protected] 2001-02-03 04:05:07.000 +07:00 - 2001-02-03 04:05:07.000 +07:00
initialize repo
"###);

// Abandon a certain operation. It wouldn't be useful, but works.
test_env.jj_cmd_ok(&repo_path, &["commit", "-m", "commit 3"]);
test_env.jj_cmd_ok(&repo_path, &["commit", "-m", "commit 4"]);
let (_stdout, stderr) = test_env.jj_cmd_ok(&repo_path, &["op", "abandon", "@--..@-"]);
insta::assert_snapshot!(stderr, @r###"
Abandoned 1 operations and reparented 1 descendant operations.
"###);
insta::assert_snapshot!(test_env.jj_cmd_success(&repo_path, &["op", "log"]), @r###"
@ 50fc6d9ddaad [email protected] 2001-02-03 04:05:15.000 +07:00 - 2001-02-03 04:05:15.000 +07:00
│ commit 7e8c543dd4b96597050139a02615492f84b29b0a
│ args: jj commit -m 'commit 4'
◉ fb5252a68411 [email protected] 2001-02-03 04:05:09.000 +07:00 - 2001-02-03 04:05:09.000 +07:00
│ commit a8ac27b29a157ae7dabc0deb524df68823505730
│ args: jj commit -m 'commit 2'
◉ f1c462c494be [email protected] 2001-02-03 04:05:07.000 +07:00 - 2001-02-03 04:05:07.000 +07:00
initialize repo
"###);

// Can't abandon the whole operation history.
let stderr = test_env.jj_cmd_failure(&repo_path, &["op", "abandon", "..@"]);
insta::assert_snapshot!(stderr, @r###"
Error: Cannot roll back to unrelated repository state
Hint: Run `jj op restore` first to move to the desired state
"###);

// Abandon empty range.
let (_stdout, stderr) = test_env.jj_cmd_ok(&repo_path, &["op", "abandon", "@..@"]);
insta::assert_snapshot!(stderr, @r###"
Abandoned 0 operations.
"###);
insta::assert_snapshot!(test_env.jj_cmd_success(&repo_path, &["op", "log", "-l1"]), @r###"
@ 50fc6d9ddaad [email protected] 2001-02-03 04:05:15.000 +07:00 - 2001-02-03 04:05:15.000 +07:00
│ commit 7e8c543dd4b96597050139a02615492f84b29b0a
│ args: jj commit -m 'commit 4'
"###);
}

#[test]
fn test_op_abandon_descendants() {
let test_env = TestEnvironment::default();
test_env.jj_cmd_ok(test_env.env_root(), &["init", "repo", "--git"]);
let repo_path = test_env.env_root().join("repo");

test_env.jj_cmd_ok(&repo_path, &["commit", "-m", "commit 1"]);
test_env.jj_cmd_ok(&repo_path, &["commit", "-m", "commit 2"]);
insta::assert_snapshot!(test_env.jj_cmd_success(&repo_path, &["op", "log"]), @r###"
@ bacc8030a969 [email protected] 2001-02-03 04:05:09.000 +07:00 - 2001-02-03 04:05:09.000 +07:00
│ commit a8ac27b29a157ae7dabc0deb524df68823505730
│ args: jj commit -m 'commit 2'
◉ bb26fe31d66f [email protected] 2001-02-03 04:05:08.000 +07:00 - 2001-02-03 04:05:08.000 +07:00
│ commit 230dd059e1b059aefc0da06a2e5a7dbf22362f22
│ args: jj commit -m 'commit 1'
◉ 19b8089fc78b [email protected] 2001-02-03 04:05:07.000 +07:00 - 2001-02-03 04:05:07.000 +07:00
│ add workspace 'default'
◉ f1c462c494be [email protected] 2001-02-03 04:05:07.000 +07:00 - 2001-02-03 04:05:07.000 +07:00
initialize repo
"###);

// For safety, can't abandon without restoring to the destination view
let stderr = test_env.jj_cmd_failure(&repo_path, &["op", "abandon", "@-.."]);
insta::assert_snapshot!(stderr, @r###"
Error: Cannot roll back to unrelated repository state
Hint: Run `jj op restore` first to move to the desired state
"###);

// Abandon recent operations. The working-copy operation id should be updated.
test_env.jj_cmd_ok(&repo_path, &["op", "restore", "@-"]);
let (_stdout, stderr) = test_env.jj_cmd_ok(&repo_path, &["op", "abandon", "@--.."]);
insta::assert_snapshot!(stderr, @r###"
Abandoned 2 operations.
"###);
insta::assert_snapshot!(test_env.jj_cmd_success(&repo_path, &["debug", "workingcopy"]), @r###"
Current operation: OperationId("bb26fe31d66f320e6d2bda90b64a99ad95d54f1ea6ecb2e61af718fcdce91668a9edf4c5b2fe9142c43ce8037f05752b48b052fb2018d667c43d38bf76b098c2")
Current tree: Merge(Resolved(TreeId("4b825dc642cb6eb9a060e54bf8d69288fbee4904")))
"###);
insta::assert_snapshot!(test_env.jj_cmd_success(&repo_path, &["op", "log"]), @r###"
@ bb26fe31d66f [email protected] 2001-02-03 04:05:08.000 +07:00 - 2001-02-03 04:05:08.000 +07:00
│ commit 230dd059e1b059aefc0da06a2e5a7dbf22362f22
│ args: jj commit -m 'commit 1'
◉ 19b8089fc78b [email protected] 2001-02-03 04:05:07.000 +07:00 - 2001-02-03 04:05:07.000 +07:00
│ add workspace 'default'
◉ f1c462c494be [email protected] 2001-02-03 04:05:07.000 +07:00 - 2001-02-03 04:05:07.000 +07:00
initialize repo
"###);
}

fn get_log_output(test_env: &TestEnvironment, repo_path: &Path, op_id: &str) -> String {
test_env.jj_cmd_success(
repo_path,
Expand Down

0 comments on commit f977926

Please sign in to comment.