[fxfs] Improve test coverage around lsm trees and pending iterators.

The focus of these tests is around:
 * cmp_upper_bound (tie breaking on start of ranges)
 * cmp_lower_bound (tie breaking on end of ranges)

Note that this doesn't address any issues yet.

Bug: b/510925696
Change-Id: I84b40fa64acfd142664521966ff72382510b852c
Reviewed-on: https://fuchsia-review.googlesource.com/c/fuchsia/+/1603988
Commit-Queue: Aaron Drew <ripper@google.com>
Reviewed-by: Chris Suter <csuter@google.com>