Subsequent RewriteRules don't transform elements added in previous transform.

[ndelargy]

import scala.xml._
import scala.xml.transform.{RewriteRule, RuleTransformer}

object TransformIssue {
  def addNewElement(): RewriteRule = new RewriteRule {
    override def transform(n: Node): Seq[Node] = n match {
      case <element></element> => <element><new></new></element>
    }
  }

  def addThingElement(): RewriteRule = new RewriteRule {
    override def transform(n: Node): Seq[Node] = n match {
      case <element>{ children@_*}</element> => <element>{ children  }</element>
      case <new></new> => <new><thing></thing></new>
    }
  }

  def change(node: Node): Node =
    new RuleTransformer(
      addNewElement(),
      addThingElement()
    ).transform(node).head

  def changeWorkaround(node: Node): Node = {
    val out1 = new RuleTransformer(
      addNewElement()
    ).transform(node).head

    new RuleTransformer(
      addThingElement()
    ).transform(out1).head
  }

}

import org.scalatest.{FlatSpec, FunSpec}
import org.scalatest._

class TransformIssueSpec extends FlatSpec with Matchers {

  it should "apply transform to created elements" in {
    val output = TransformIssue.change(<element></element>)
    output should be(<element><new><thing></thing></new></element>)
  } // fails

  it should "work the same as the workaround imo" in {
    TransformIssue.change(<element></element>) should equal(TransformIssue.changeWorkaround(<element></element>))
  } // fails

}

When we apply a transform with two rewrite rules: the first one adding a new element, the second one adding children to the new element; then the second rewrite rule does not match on the elements added in the first rule.

When we apply the same RewriteRules in two separate RuleTransformers it does add the children to the elements added in the first step. We would expect the change and changeWorkaround functions to produce the same output.

[ashawley]

Indeed, this is a known issue. On the Getting started page, it is written:

Keep in mind that rewrite rules won't compose if they modify children, or modify values that other rewrite rules depend on.

There's a chance that RuleTransformer executes rewrite rules backward. What if you try changing the order?

    new RuleTransformer(
      addThingElement(),
      addNewElement()
    )

If that's not it, then indeed, it's probably that you need to call the transformer on the new elements yourself:

...
      case <element>{ children@_*}</element> => <element>{ change(children)  }</element>

[ashawley]

@ndelargy I hope I was able to help and you were able to make progress. Thanks you for giving a simple example case.

[ashawley]

I'll keep this open, since it would be helpful if someone investigated whether fixing this is possible.

[ndelargy]

Hi @ashawley thanks for getting back so quick on this...
We did try reversing the order of the RewriteRules but this didn't fix the issue.

[dcsobral]

The problem is that RuleTransformer recurses before applying the rules, but not after, so any rules dependending on the recursion to be applied will not work on anything added by previous rules.

However, following rules will apply on the way back from recursion. For example:

  def addNewElement(): RewriteRule = new RewriteRule {
    override def transform(n: Node): Seq[Node] = {
      n match {
        case <thing>{ children @ _* }</thing> => <new><thing>{ children }</thing></new>
        case other => other
      }
    }
  }

  def addThingElement(): RewriteRule = new RewriteRule {
    override def transform(n: Node): Seq[Node] = {
      n match {
        case <new>{ children @ _*}</new> => <element><new>{ children }</new></element>
        case other => other
      }
    }
  }

If you call TransformIssue.change(<thing/>), it will return <element><new><thing/></new></element>.

If RuleTransformer was written like this, it would pass the tests:

class RuleTransformer(rules: RewriteRule*) extends BasicTransformer {
    override def transform(n: Node): Seq[Node] = {
        rules.foldLeft(n: Seq[Node]) { (res, rule) => rule transform res.flatMap(super.transform) }
    }
}

However, it would through a stack overflow exception on the example I gave. The only solution I see is break the relationship between RuleTransformer and BasicTransformer, like this:

class NestingTransformer(rule: RewriteRule) extends BasicTransformer {
  override def transform(n: Node): Seq[Node] = {
    rule.transform(super.transform(n))
  }
}

class RuleTransformer(rules: RewriteRule*) {
  val nestingTransformers = rules.map(new NestingTransformer(_))
  def transform(n: Node): Seq[Node] = {
    nestingTransformers.foldLeft(n: Seq[Node]) { (res, transformer) =>
      transformer.transform(res)
    }
  }
}

Fun fact: RuleTransformer above is exactly what the workaround is, while NestingTransformer is what RuleTransfomer was but without the ability to process multiple rules.

I can't see a case for the exact semantics that RuleTransformer provides. It doesn't prevent rewrites from seeing previous rewrites -- the example would work if written <element>{ transform(children) }</element> -- just make it so it won't recurse to see those changes.