Quantcast
Channel: Advanced Reactive Java
Viewing all 49 articles
Browse latest View live

Operator concurrency primitives: subscription-containers (part 2)

$
0
0

Introduction


In this blog post, I'm going to implement two, lock-free versions of the TwoSubscriptions container from the previous post. Although they will be functionally equivalent, the implementation will reflect two different philosophies regarding how one can check for their unsubscription status and unsubscribe them.


Using boolean isUnsubscribed in the state


A simple way of implementing a lock-free data structure is to do a so-called copy-on-write operation on every mutation, which involves an immutable state and a CAS loop. With our TwoSubscriptions, we will capture the state of two distinct Subscriptions into a composite class:

    static final class State {
final Subscription s1;
final Subscription s2;
final boolean isUnsubscribed;
public State(Subscription s1,
Subscription s2,
boolean isUnsubscribed) {
this.s1 = s1;
this.s2 = s2;
this.isUnsubscribed = isUnsubscribed;

}
}
// ...

With this State inner class, whenever the state needs to change, we will create a new instance, copy over the relevant values and use a CAS loop to achieve atomicity. Now let's see our new lock-free container's class structure:

public final class TwoSubscribersLockFree1 
implements Subscription {
static final class State {
// ...
}

static final State EMPTY =
new State(null, null, false); // (1)

static final State UNSUBSCRIBED =
new State(null, null, true); // (2)

final AtomicReference<State> state =
new AtomicReference<>(EMPTY); // (3)

public void set(boolean first, Subscription s) {
// implement
}

@Override
public void unsubscribe() {
// implement
}

@Override
public boolean isUnsubscribed() {
// implement
}
}

First, since the initial and terminal states are essentially constants, they are declared as static final instances with the difference that UNSUBSCRIBED.isUnsubscribed == true (1) (2). Since the state needs to be changed atomically, we also need an AtomicReference to hold the State instance (3) which we initialize to the empty (constant) state.

With the given skeleton, the implementation of set() looks as follows:

    public void set(boolean first, Subscription s) {
for (;;) {
State current = state.get(); // (1)
if (current.isUnsubscribed) { // (2)
s.unsubscribe();
return;
}
State next;
Subscription old;
if (first) {
next = new State(s, current.s2, false); // (3)
old = current.s1; // (4)
} else {
next = new State(current.s1, s, false);
old = current.s2;
}
if (state.compareAndSet(current, next)) { // (5)
if (old != null) {
old.unsubscribe(); // (6)
}
return;
}
}
}

and works as follows:

  1. The current state value is read.
  2. If the current state is unsubscribed, the terminal state of this container is was reached, we unsubscribe the parameter and quit.
  3. Otherwise, we'll create a new state based on the old one, replace the appropriate subscription with the provided one.
  4. Since the subscription needs to be unsubscribed on replacement, we save its instance locally.
  5. The CAS operation will atomically swap in the new, updated state or we perform a new iteration in case a concurrent modification happened to the state.
  6. With a successful CAS, the original subscription (if any) is unsubscribed and the loop is quit.
The implementation of isUnsubscribed() is straightforward:

    // ...
@Override
public boolean isUnsubscribed() {
return state.get().isUnsubscribed;
}
// ...

Finally, let's see how one can implement the unsubscribe() method.

    @Override
public void unsubscribe() {
State current = state.get(); // (1)
if (!current.isUnsubscribed) { // (2)
current = state.getAndSet(UNSUBSCRIBED); // (3)
if (!current.isUnsubscribed) { // (4)
List<Throwable> errors = null; // (5)

errors = unsubscribe(current.s1, errors); // (6)
errors = unsubscribe(current.s2, errors);

Exceptions.throwIfAny(errors); // (7)
}
}
}

private List<Throwable> unsubscribe(Subscription s, // (8)
List<Throwable> errors) {
if (s != null) {
try {
s.unsubscribe();
} catch (Throwable e) {
if (errors == null) {
errors = new ArrayList<>();
}
errors.add(e);
}
}
return errors;
}
}

The method has several interesting steps:

  1. We retrieve the current state.
  2. If the current state is already unsubscribed, there is nothing to do and the method quits.
  3. Otherwise, we atomically exchange the current state with the constant terminal state.
  4. If the previous state was unsubscribed, the method can quit, otherwise, since the getAndSet is atomic there will be exactly one caller who transitions from a non-terminated state into the terminated state. There is no need for a CAS loop here and the unsubscription, so far, can be wait-free on platforms with intrinsified getAndSet.
  5. The possible exceptions are collected into an errors list.
  6. I've factored out the unsubscription and error collection into a method and it is called for each of the contained subscriptions.
  7. If any of the unsubscriptions threw, the exception(s) are rethrown.
  8. The convenience method of unsubscribing a subscription and updating the errors list if necessary.

Using the UNSUBSCRIBED state reference

If we think about it, since the terminal state is distinct from the others not just by the isUnsubscribed flag, but by having a unique constant reference. It is possible remove isUnsubscribed and compare against the UNSUBSCRIBED instance everywhere as necessary.

Therefore, we can simplify the State class in the new TwoSubscribersLockFree2 as follows:

public final class TwoSubscribersLockFree2 implements Subscription {
static final class State {
final Subscription s1;
final Subscription s2;
public State(Subscription s1,
Subscription s2) {
this.s1 = s1;
this.s2 = s2;

}
}

The isUnsubscribed field was removed from it and we have to change every former isUnsubscribed check:

    // ...
static final State EMPTY = new State(null, null); // (1)
static final State UNSUBSCRIBED = new State(null, null);

final AtomicReference<tate> state
= new AtomicReference<>(EMPTY);

public void set(boolean first, Subscription s) {
for (;;) {
State current = state.get();
if (current == UNSUBSCRIBED) { // (2)
s.unsubscribe();
return;
}
State next;
Subscription old;
if (first) {
next = new State(s, current.s2);
old = current.s1;
} else {
next = new State(current.s1, s);
old = current.s2;
}
if (state.compareAndSet(current, next)) {
if (old != null) {
old.unsubscribe();
}
return;
}
}
}

@Override
public boolean isUnsubscribed() {
return state.get() == UNSUBSCRIBED; // (3)
}

@Override
public void unsubscribe() {
State current = state.get();
if (current != UNSUBSCRIBED) { // (4)
current = state.getAndSet(UNSUBSCRIBED);
if (current != UNSUBSCRIBED) { // (5)
List<Throwable> errors = null;

errors = unsubscribe(current.s1, errors);
errors = unsubscribe(current.s2, errors);

Exceptions.throwIfAny(errors);
}
}
}
// ...

The new constants no longer need a boolean flag (1) and places of current.isUnsubscribed are now replaced with current == UNSUBSCRIBED check (2, 3, 4, 5).

Given these two approaches, which one to choose? Benchmark and see it for yourself. Obviously, the first allocates more memory but the boolean check can be faster on certain platforms, whereas the second costs less in memory but reference comparison can be slower.

Generally though, using the class will increase the GC pressure as every modification triggers the allocation of a new state. It is possible to avoid it by performing per-subscription CAS loops, but the approach can get cumbersome as the number of subscription fields increases.

Conclusion

In this post, I've introduced two lock-free variants of the TwoSubscriptions container and explained their inner workings.

It is more likely one has to manage more than two subscriptions at a time, therefore, I'm going to demonstrate an array-based container with the very same underlying approaches in the next post.


Operator concurrency primitives: subscription-containers (part 3 - final)

$
0
0

Introduction


In this final part about subscription-containers, I'm going to demonstrate an array-backed container based on the copy-on-write approach and atomics.

Why is this type of container so important? My answer is another question: if the contained subscriptions were Subscribers, what would you do with an array of Subscribers?

You could implement operators requiring multicast to child subscribers with it and maintain the thread-safety and termination guarantees, similar to how RxJava's Subjects handle their subscribers and more importantly, how the - recently rewritten - publish() does it too.


Array-backed container

Let's build such a container with the following requirements: ability to add and remove some kind of subscriptions (i.e., Subscribers), ability to get the current contents, ability to 'unsubscribe' the container without unsubscribing the contents and the ability to tell if an add succeeded or not.

The class skeleton, with the trivial implementations already filled in, looks like this:


@SuppressWarnings({"rawtypes", "unchecked"})                 // (1)
public class SubscriberContainer<T> {
static final Subscriber[] EMPTY = new Subscriber[0]; // (2)
static final Subscriber[] TERMINATE = new Subscriber[0];

final AtomicReference<Subscriber[]> array
= new AtomicReference<>(EMPTY); // (3)

public Subscriber<T>[] get() { // (4)
return array.get();
}

public boolean add(Subscriber<T> s) { // (5)
// implement
}

public boolean remove(Subscriber<T> s) { // (6)
// implement
}

public Subscriber<T>[] getAndTerminate() { // (7)
return array.getAndSet(TERMINATE);
}

public boolean isTerminated() { // (8)
return get() == TERMINATED;
}
}

The structure consists of elements as follows:

  1. Java doesn't allow generic base types for arrays and doesn't like certain type conversions so we are forced to suppress the warnings regarding raw types and converting those raw values back to parametric types. Generally, the implementation has to be safe but the user of the class will remain typesafe.
  2. We use a constant array for indicating the empty state and the terminated state.
  3. We store the current array in an AtomicReference instance, but if you know the class won't be extended further, you can just extend AtomicReference directly.
  4. The get() method returns the current array of contained items. For performance reasons, one should use this array in read-only manner (otherwise, defensive copies are required on each invocation).
  5. The add() method takes a properly-typed Subscriber and returns true if it could be added or false if the container was terminated.
  6. The remove() method tries to remove the given subscriber instance and returns true if successful.
  7. Instead of a plain unsubscribe() method, we do a handy trick: the current array is replaced with the terminal indicator and return the previous array. It becomes useful when an atomic termination and post-termination actions are needed.
  8. Since a regular empty array can't be told apart from the terminated state, a method is needed which checks for this terminal state explicitly.
The add() method will be the simpler one:

    // ...
public boolean add(Subscriber<T> s) {
for (;;) {
Subscriber[] current = array.get();
if (current == TERMINATED) { // (1)
return false;
}
int n = current.length;
Subscriber[] next = new Subscriber[n + 1];
System.arraycopy(current, 0, next, 0, n); // (2)
next[n] = s;
if (array.compareAndSet(current, next)) { // (3)
return true;
}
}
}
// ...

Here we have a classical CAS loop:

  1. If the container contains the terminated token, the method returns with false and the subscriber is not added to the container. The caller can then decide what to do with said subscriber (i.e., send onCompleted event to it).
  2. A copy of the current contents are made into an array with length + 1 and the subscriber is assigned to the very end.
  3. The CAS will try to 'commit' the changes and the method returns true. Otherwise, a new attempt is made.

Lastly, the remove() looks like as follows:

    // ...
public boolean remove(Subscriber<T> s) {
for (;;) {
Subscriber[] current = array.get();
if (current == EMPTY
|| current == TERMINATED) { // (1)
return false;
}
int n = current.length;
int j = -1;
for (int i = 0; i < n; i++) { // (2)
Subscriber e = current[i];
if (e.equals(s)) {
j = i;
break;
}
i++;
}
if (j < 0) { // (3)
return false;
}
Subscriber[] next;
if (n == 1) { // (4)
next = EMPTY;
} else {
next = new Subscriber[n - 1];
System.arraycopy(current, 0, next, 0, j);
System.arraycopy(current, j + 1,
next, j, n - j - 1); // (5)
}
if (array.compareAndSet(current, next)) { // (6)
return true;
}
}
}

Although a bit complicated, the method's behavior is straightforward:

  1. In case the current array is empty or the container is already terminated, it can't contain any subscribers and thus the method quits immediately.
  2. Otherwise, we search for the first occurrence of the given subscriber and hold onto its index in j. By scanning first instead of doing an on-the-fly filter-copy, we can save some overhead due to the card-marking associated with each reference store, required by most GCs.
  3. If j remained negative, the subscriber is not amongst the others in the array and the method returns false.
  4. In case the array contains a single value, there is no need to create an empty array but we can reuse the constant (since empty arrays are essentially stateless).
  5. If the array contains multiple elements, a new shorter array is created and values around the found location is then copied over to it.
  6. Finally, the CAS will swap in the new array and returns true indicating the subscriber was successfully added.

Such a container is quite often used in multicast-like operators but these operators rarely encounter more than half-dozen subscribers during their lifecycle and as such, the frequent allocation of arrays has less impact.

If the allocation rate is a problem in your scenario, it is possible to change the above logic to use synchronized block and some type of list or set to store the subscriptions, but note that the get() method, quite frequently used when dispatching events, can no longer be implemented in a wait-free manner. The get() method will, most likely, require the use of a synchronized block and making defensive copies for every invocation; again, a tradeoff one needs to consider carefully.


Conclusion

In this mini-series, I talked about various kinds of standard subscription containers and shown how to implement blocking and lock-free containers. In addition, I've described an array-backed lock-free container with peculiar properties that come in handy when implementing multicast-like operators.

If there was only RxJava 1.x, the operator concurrency primitives series could end here. However, the reactive-streams has been recently finalized and is expected to form the basis for RxJava 2.0 and that is going to be a complete rewrite; it is unavoidable.

Does this mean the things learned so far are useless? Yes and no. The concepts will be quite relevant in fact, only the class structures need to change somewhat to match the reactive-streams specification and requirements.

Instead of jumping right into RxJava 2.0 constructs, let's take a break and look at some other advanced RxJava topics: schedulers

Schedulers (part 1)

$
0
0

Introduction

Schedulers are the key to asynchronous and concurrent computations in RxJava. Although many standard schedulers exist and you can wrap an Executor into a scheduler, it is worth understanding how schedulers can be built from scratch in order to utilize other forms of concurrency sources, such as GUI event loops of frameworks.


The Scheduler API

If you are familiar with Rx.NET's IScheduler API, you can discover that RxJava's Scheduler API is a bit different. This difference comes from how each library tried to solve the recursive scheduling problem. Rx.NET chose to inject the actual scheduler into the action being scheduled.

RxJava chose to mirror how Iterable/Iterator pattern is established and came up with the pair of Scheduler/Worker. RxJava's scheduler doesn't do any scheduling but allows the creation of a Worker, which allows scheduling directly and recursively and is safe to close-over in the submitted action if necessary. In addition, the Worker itself is a Subscription and can be unsubscribed which triggers a mass-unsubscription and prevents any further tasks getting scheduled (best effort I might add). This comes in handy when the scheduling is used in operators (for example, buffer with time) and the downstream unsubscribes the whole chain, cancelling the periodic buffer-emission task once and for all.

The Scheduler/Worker API has to meet some requirements:

  1. All methods should be thread-safe.
  2. Workers should make sure undelayed, sequentially scheduled tasks execute in FIFO order.
  3. Workers must make best effort to cancel outstanding tasks when unsubscribed.
  4. Unsubscription of a Worker should not affect other Worker instances of the same Scheduler.

These requirements may seem harsh, but allows reasoning about a dataflow's concurrency much easier, similar to how the sequential requirement of the Observer methods allows the same reasoning.

In addition to the requirements, there are a few nice-to-have properties associated with the API:

  1. Tasks scheduled on a Worker should try to avoid hopping threads. (Thread locality improves performance.)
  2. Delayed tasks scheduled sequentially and with the same delay amount should keep their FIFO order (with concurrent scheduling, the ordering bets are off).
With all these requirements, a conservative implementation is likely to use a single-threaded thread pool backing each individual worker and this is how the standard RxJava schedulers are implemented: the underlying ScheduledExecutorService gives all of these guarantees.



Implementing a custom Scheduler

Let's assume we need to write a custom scheduler with the following properties: (1) it should only have a single worker thread and (2) a thread-local context value needs to be 'transferred over' and made available to the executing task via the same thread-local access mechanism.

Clearly, if we'd have only (1) as the property, one could just wrap a single-threaded executor with Executors.from(), but property (2) requires some additional work to be performed when a task is prepared and executed.

In order to accomplish the requirements, we may reuse some of RxJava's own scheduler primitives: namely ScheduledAction and NewThreadWorker. (Note however, that these are internal classes and are subject to changes without warnings. Here, I'm using them to reduce the clutter and allow me to concentrate on the important parts of creating the scheduler.)

As usual, we start with the class skeleton:


public final class ContextAwareScheduler 
extends Scheduler {

public static final ContextAwareScheduler INSTANCE =
new ContextAwareScheduler(); // (1)

final NewThreadWorker worker;

private ContextAwareScheduler() {
this.worker = new NewThreadWorker(
new RxThreadFactory("ContextAwareScheduler")); // (2)
}
@Override
public Worker createWorker() {
return new ContextAwareWorker(worker); // (3)
}

static final class ContextAwareWorker extends Worker {

final CompositeSubscription tracking; // (4)
final NewThreadWorker worker;

public ContextAwareWorker(NewThreadWorker worker) {
this.worker = worker;
this.tracking = new CompositeSubscription();
}
@Override
public Subscription schedule(Action0 action) {
// implement
}
@Override
public Subscription schedule(Action0 action,
long delayTime, TimeUnit unit) {
// implement
}
@Override
public boolean isUnsubscribed() {
return tracking.isUnsubscribed(); // (5)
}
@Override
public void unsubscribe() {
tracking.unsubscribe();
}
}
}

Our ContextAwareScheduler skeleton may look scary, but it consists of straightforward components:

  1. Since we want a single global thread, we can't allow multiple instances of the Scheduler to exist and thus we use a static instance variable for it.
  2. The scheduler will delegate most of its work to a single underlying worker. We reuse the NewThreadWorker class and the RxThreadFactory to get a single daemon-thread backed worker instance.
  3. Instead of handing out the single worker, we need to 'split' it among usages to conform to requirement #4. Otherwise, if the worker is unsubscribed, everyone's worker is now unsubscribed and useless from then on.
  4. We still need to make sure requirement #3 is met, therefore, we need to track tasks submitted to each particular worker.
  5. The tracking structure also gives the means to check for and issue unsubscriptions.
Next, we need the aforementioned thread-local context:

public final class ContextManager {
static final ThreadLocal<Object> ctx = new ThreadLocal<>();

private ContextManager() {
throw new IllegalStateException();
}

public static Object get() {
return ctx.get();
}
public static void set(Object context) {
ctx.set(context);
}
}

The ContextManager just wraps around a static ThreadLocal instance. In practice, you'd want to replace the Object with some meaningful type.

Now back to the implementation of the schedule() methods:


    // ...
@Override
public Subscription schedule(Action0 action) {
return schedule(action, 0, null); // (1)
}
@Override
public Subscription schedule(Action0 action,
long delayTime, TimeUnit unit) {

if (isUnsubscribed()) { // (2)
return Subscriptions.unsubscribed();
}

Object context = ContextManager.get(); // (3)
Action0 a = () -> {
ContextManager.set(context); // (4)
action.call();
};

return worker.scheduleActual(a,
delayTime, unit, tracking); // (5)
}
// ...

Just look at the simplicity!

  1. We delegate the undelayed scheduling as if the initial delay is zero. All underlying structure will interpret this zero correctly and perform the required undelayed FIFO execution.
  2. In case the current worker is unsubscribed, we don't do anything and return a constant unsubscribed subscription. Note that unsubscribing always involves some small race window where tasks (or events) may slip through. This race is internally resolved in the scheduleActual; more on this below.
  3. We hold onto the current thread's context value and wrap the action into another action. 
  4. Inside, the context value is restored to that particular thread's local storage. Since the worker is single threaded and non-reentrant, the thread-local context can't be overwritten by the next scheduled task while a previous task is running.
  5. Finally, we delegate the wrapping action and the delay information to the underlying NewThreadWorker instance. By passing in the tracking composite, the action will be properly tracked and removed if the task completes, gets unsubscribed or the entire worker gets unsubscribed.


As mentioned in the explanation steps, step (2) is inherently racing with the unsubscription of the worker but we shouldn't leave tasks behind after unsubscription. This is where the unsubscription guarantee of a container comes into play. If we wrap the Future returned by the underlying thread pool into a Subscription, we can safely add it to the tracking composite which will either atomically take a hold onto it or unsubscribe it immediately.

Let's try it out:

Worker w = INSTANCE.createWorker();

CountDownLatch cdl = new CountDownLatch(1);

ContextManager.set(1);
w.schedule(() -> {
System.out.println(Thread.currentThread());
System.out.println(ContextManager.get());
});

ContextManager.set(2);
w.schedule(() -> {
System.out.println(Thread.currentThread());
System.out.println(ContextManager.get());
cdl.countDown();
});

cdl.await();

ContextManager.set(3);

Observable.timer(500, TimeUnit.MILLISECONDS, INSTANCE)
.doOnNext(v -> {
System.out.println(Thread.currentThread());
System.out.println(ContextManager.get());
}).toBlocking().first();

w.unsubscribe();

Conclusion

Schedulers give the opportunity to specify where and likely when to execute tasks related to the operation of an Observable chain. Built-in schedulers should cover most of the usual needs towards parametric concurrency, but some scenarios require the use and building of custom schedulers. In this blog post, I've shown how one can, with the major help from existing RxJava classes, build his/her own custom scheduler with a custom behavior.

In the next part, we are going in deeper and look at the ScheduledAction class and see how its concepts can be utilized in case more control needs to be enforced when scheduling a task, for example, to work with or against thread interruptions.

Pitfalls of operator implementations (part 3)

$
0
0

Introduction

In this series, I put a spotlight onto some common and some less common yet sneaky pitfalls of operator implementations. Now that we know more about producers, subscription-containers and schedulers, let's see some more pitfalls.

#9: Subscribing twice

Some operators, especially those built upon OnSubscribe may take their Subscribers and subscribe them to another Observable. An example for such an operator is defer().

Let's assume you want to create an operator which calls an action callback before it subscribes the incoming Subscriber to the real Observable:

public final class OnSubscribeRunAction<T> 
implements OnSubscribe<T> {
final Observable actual;
final Action0 action;
public OnSubscribeRunAction(Observable actual, Action0 action) {
this.action = action;
this.actual = actual;
}
@Override
public void call(Subscriber child) {
try {
action.call();
} catch (Throwable e) {
child.onError(e);
return;
}
actual.unsafeSubscribe(child);
}
}

Observable<Integer> source = Observable.create(
new OnSubscribeRunAction<>(Observable.range(1, 3),
() -> {
System.out.println("Subscribing!");
}));

TestSubscriber<Integer> ts = new TestSubscriber<Integer>() {
@Override
public void onStart() {
Thread t = new Thread(() -> {
System.out.println("Starting helper thread "
+ Thread.currentThread());
});
t.start();
}
};
source.unsafeSubscribe(ts);

If we run the example code, we see that onStart is called twice! The problem is that how the backpressure-related logic is designed in RxJava: whenever a child is subscribed, its onStart() method is called which allows the client to get some code executed before the first onNext value. Usually, this is where the initial request amount is issued or perhaps a GUI window associated with the subscriber is open.

Now in regular end-subscribers, this rarely comes up because the subscribe() method wraps them into a SafeSubscriber which doesn't forward its onStart method. However, when dealing with one another's operators, unsafeSubscribe is very common and the onStart ends up being called multiple times.

The resolution is to wrap the child subscriber with another subscriber in the operator that doesn't forward the onStart method:


// ...
try {
action.call();
} catch (Throwable e) {
child.onError(e);
return;
}
actual.unsafeSubscribe(new Subscriber<T>(child) {
@Override
public void onNext(T t) {
child.onNext(t);
}
@Override
public void onError(Throwable e) {
child.onError(e);
}
@Override
public void onCompleted() {
child.onCompleted();
}
});
// ...


#10: Leaking scheduler workers

Let's say instead of performing some arbitrary action on subscription immediately, one would like to delay its execution by some time. If the sequence doesn't complete in time, one can perform some mitigating actions (i.e., showing a work in progress dialog for example).

public final class OnSubscribeRunActionDelayed<T>
implements OnSubscribe<T> {
final Observable actual;
final Action0 action;
final long delay;
final TimeUnit unit;
final Scheduler scheduler;
public OnSubscribeRunActionDelayed(Observable actual,
Action0 action, long delay,
TimeUnit unit, Scheduler scheduler) {
this.action = action;
this.actual = actual;
this.delay = delay;
this.unit = unit;
this.scheduler = scheduler;
}
@Override
public void call(Subscriber<? super T> child) {
SerializedSubscriber<T> s =
new SerializedSubscriber<>(child);

Worker w = scheduler.createWorker(); // (1)

Subscription cancel = w.schedule(() -> {
try {
action.call();
} catch (Throwable e) {
s.onError(e);
}
}, delay, unit);

actual
.doOnCompleted(cancel::unsubscribe)
.unsafeSubscribe(s);
}
}

Observable<Integer> source = Observable.create(
new OnSubscribeRunActionDelayed<>(Observable
.just(1).delay(1, TimeUnit.SECONDS),
() -> {
System.out.println("Sorry, it takes too long...");
}, 500, TimeUnit.MILLISECONDS, Schedulers.io()));

Subscription s = source.subscribe(System.out::println);

Thread.sleep(250);

s.unsubscribe();

Thread.sleep(1000);

source.subscribe(System.out::println);

Thread.sleep(1500);

for (Thread t : Thread.getAllStackTraces().keySet()) {
if (t.getName().startsWith("RxCached")) {
System.out.println(t);
}
}
}
Again, running the example gives unwanted results: the excuse message is printed even if the first subscription was cancelled and when we dump the threads at the end, we'll see two RxCachedThreadSchedulers but clearly only one should be there due to reusability.

The problem is that the worker and the schedule token is not participating in unsubscription properly: even if the actual Observable is fast, only the work is unsubscribed but the worker is not, and thus it is never returned to the cache-pool.

The bug is sneaky because Schedulers.computation() and Schedulers.trampoline() are not sensitive to scheduler leaks: the former arbitrates between a fixed set of actual workers and the latter doesn't retain any threading resources and can be cleanly garbage collected. Schedulers.io(), Schedulers.from() and newThread(), on the other hand, hold onto a thread which can't be reused / shutdown unless the worker is unsubscribed.

The solution is to add the worker and the cancel token to the child subscriber as resource so they are unsubscribed if the child unsubscribes, however, since there is going to be a single scheduled task, unsubscribing the worker will unsubscribe 'all' of the pending and running tasks thus there is no need to add the individual cancel token to the child subscriber; the adding the worker is enough:

    // ...
SerializedSubscriber<T> s = new SerializedSubscriber<>(child);

Worker w = scheduler.createWorker();
child.add(w);

w.schedule(() -> {
try {
action.call();
} catch (Throwable e) {
s.onError(e);
}
}, delay, unit);

actual
.doOnCompleted(w::unsubscribe)
.unsafeSubscribe(s);
// ...

#11: Adding the worker to the subscriber

Let's assume we need an operator that when receives a value, it emits an observable that will emit that single value after some delay.

public final class ValueDelayer<T> 
implements Operator<Observable<T>, T> {
final Scheduler scheduler;
final long delay;
final TimeUnit unit;

public ValueDelayer(long delay,
TimeUnit unit, Scheduler scheduler) {
this.delay = delay;
this.unit = unit;
this.scheduler = scheduler;
}

@Override
public Subscriber<? super T> call(
Subscriber<? super Observable<T>> child) {
Worker w = scheduler.createWorker();
child.add(w);

Subscriber<T> parent = new Subscriber<T>(child, false) {
@Override
public void onNext(T t) {
BufferUntilSubscriber<T> bus =
BufferUntilSubscriber.create();

w.schedule(() -> {
bus.onNext(t);
bus.onCompleted();
}, delay, unit);

child.onNext(bus);
}
@Override
public void onError(Throwable e) {
child.onError(e);
}
@Override
public void onCompleted() {
child.onCompleted();
}
};

child.add(parent);

return parent;
}
}

Observable.range(1, 3)
.lift(new ValueDelayer<>(1, TimeUnit.SECONDS,
Schedulers.computation()))
.take(1)
.doOnNext(v -> v.subscribe(System.out::println))
.subscribe();

Thread.sleep(1500);

Strangely, the example prints nothing but we expect it to print 1 after a second. The problem is with the take(1) unsubscribing the upstream after receiving the first 'window' which then cancels the scheduled emission of the value itself.

Resolving the problem can take many shapes and actually depends on the broader context. Clearly, we need to unsubscribe the worker yet allow the consumption of the inner observable sequence.

One way is to keep an atomic counter to count the number of unobserved inner Observables and unsubscribe the worker if it reaches zero. In addition, this solution requires the inner Observables to be always consumed.

        // ...
Worker w = scheduler.createWorker();

final AtomicBoolean once = new AtomicBoolean();
final AtomicInteger wip = new AtomicInteger(1); // (1)

Subscriber<T> parent = new Subscriber<T>(child, false) {
@Override
public void onNext(T t) {

if (wip.getAndIncrement() == 0) { // (2)
wip.set(0);
return;
}

BufferUntilSubscriber<T> bus =
BufferUntilSubscriber.create();

w.schedule(() -> {
try {
bus.onNext(t);
bus.onCompleted();
} finally {
release(); // (3)
}
}, delay, unit);

child.onNext(bus);
if (child.isUnsubscribed()) {
if (once.compareAndSet(false, true)) { // (4)
release();
}
}
}
@Override
public void onError(Throwable e) {
child.onError(e);
}
@Override
public void onCompleted() {
if (once.compareAndSet(false, true)) {
release(); // (5)
}
child.onCompleted();
}
void release() {
if (wip.decrementAndGet() == 0) {
w.unsubscribe();
}
}
};
parent.add(Subscriptions.create(() -> { // (6)
if (once.compareAndSet(false, true)) {
if (wip.decrementAndGet() == 0) {
w.unsubscribe();
}
}
}));

child.add(parent);

return parent;
}
// ...

The solution involves several notable changes:

  1. We need an atomic integer and an atomic boolean. The former one counts the unconsumed inner Observables as well as the active main upstream source of Ts. Since the upstream can end at many locations and perhaps 'multiple times' (i.e., terminating just after the onNext but the upstream still sends an onCompleted which then triggers an unsubscription coming from downstream). Since the upstream should count as 1, we need to use the once to make sure the upstream's single decrement happens only once.
  2. We increment the number of 'open windows' by one before even attempting to schedule its task. However, since (6) can asynchronously decrement the wip value to zero, an increment from 0 to 1 indicates an onNext slipped through in which case the worker is already unsubscribed and the child would receive an Observable that can't ever emit.
  3. Once the individual inner Observables could emit their values, we release one 'window'.
  4. We eagerly check if the child has just unsubscribed after the emission of the inner 'window'. If so, we try to release the upstream once.
  5. If the downstream didn't unsubscribe, the onCompleted has to try and release the upstream once too.
  6. But since unsubscription can happen at any time, even between event emissions, we still have to try and release the upstream once.

Conclusion

One can argue that these pitfalls are corner cases but as an operator developer, especially if planning to submit a PR, has to look out for such problems.

In this post, we've looked at how one can accidentally start its subscribers multiple times and shown two opposite cases where adding or not-adding a worker to the child subscriber can cause problems.

Schedulers (part 2)

$
0
0

Introduction

In the previous post, I've show how one can build a custom scheduler mainly based on existing RxJava classes.

In this post, I'm going deeper and show how one can control the interaction between the underlying ExecutorService and RxJava constructs at a level which is inaccessible through NewThreadWorker.


The ScheduledAction class

In the pre-Scheduler/Worker days, interacting with a Future was straightforward; we just wrapped its cancel() method with a Subscription and they were added to various subscription-containers.

However, with the Scheduler/Worker API, this doesn't work anymore due to the need to track and mass-cancel such tasks. Once the Future's get tracked, they need to be un-tracked if they complete or get cancelled, or else face memory leaks. This extra administration means that one can't just submit an Action0/Runnable directly to the ExecutorService but he/she needs to decorate it so either the cancellation or regular completion cleans up after the task.

The solution comes in the form of the ScheduledAction class. Every regular Action0 is wrapped into these by NewThreadWorker.scheduleActual() methods. The ScheduledAction class contains a SubscriptionList as the holder for 'actions' that need to be executed on a normal completion and on general unsubscription:

public final class ScheduledAction 
implements Runnable, Subscription {
final Action0 action; // (1)
final SubscriptionList slist; // (2)

public ScheduledAction(Action0 action) {
this.action = action;
this.slist = new SubscriptionList();
}
@Override
public void run() {
try {
action.call(); // (3)
} finally {
unsubscribe(); // (4)
}
}
@Override
public boolean isUnsubscribed() {
return slist.isUnsubscribed();
}
@Override
public void unsubscribe() {
slist.unsubscribe();
}

public void add(Subscription s) { // (5)
slist.add(s);
}
}

The class is relatively straightforward:

  1. We hold onto the real action to be executed.
  2. We need a composite to store all the unsubscription actions. Since this list will be add-only, the SubscriptionList will suffice.
  3. Since the ExecutorService requires a Runnable instance, the class extends this interface and in the run() method, we delegate to the actual action.
  4. Whether or not the call succeeds, we call unsubscribe() on ourselves which should trigger the necessary cleanup actions.
  5. However, these cleanup actions need to be registered with this ScheduledAction thus we need to expose the add() method of the SubscriptionList.
The next step is to wire up all the tracking and cleanup actions before the action gets submitted to an ExecutorService. For simplicity, I'll assume said service is a single-threaded service. I'll deal with a multi-threaded service later. Let's start with the skeleton of a new custom Worker:

public final class CustomWorker 
extends Scheduler.Worker {
final ExecutorService exec; // (1)
final CompositeSubscription tracking; // (2)
final boolean shutdown; // (3)

public CustomWorker() {
exec = Executors.newSingleThreadExecutor();
tracking = new CompositeSubscription();
shutdown = true;
}
public CustomWorker(ExecutorService exec) {
this.exec = exec;
tracking = new CompositeSubscription();
shutdown = false; // (4)
}
@Override
public Subscription schedule(Action0 action) {
return schedule(action, 0, null); // (5)
}
@Override
public Subscription schedule(Action0 action,
long delayTime, TimeUnit unit) {
// implement
}
@Override
public boolean isUnsubscribed() {
return tracking.isUnsubscribed(); // (6)
}
@Override
public void unsubscribe() {
if (shutdown) {
exec.shutdownNow(); // (7)
}
tracking.unsubscribe();
}
}

At this point, the skeleton isn't complicated either:

  1. We store the reference to the actual thread pool.
  2. We also need to track the tasks submitted in order to mass-cancel them.
  3. We plan to allow users to submit their own single-threaded executor services, in which case shutting it down should be the responsibility of the caller. 
  4. We'll only shut down our own services but not those received through the constructor.
  5. We delegate the non-delayed scheduling to the delayed-scheduling via a zero delayTime value.
  6. The tracking structure doubles as the way of telling if the worker has been unsubscribed.
  7. In case the ExecutorService was created by this class, we'll shut it down, then all tracked tasks are unsubscribed. (Note that if the service is our own, we don't really need to track the tasks because the service itself already tracks all submitted tasks the call to shutdownNow() will mass-cancel them.)
Finally, let's see the implementation of the delayed schedule() method:

    // ...
@Override
public Subscription schedule(Action0 action,
long delayTime, TimeUnit unit) {
if (isUnsubscribed()) { // (1)
return Subscriptions.unsubscribed();
}
ScheduledAction sa = new ScheduledAction(action); // (2)

tracking.add(sa); // (3)
sa.add(Subscriptions.create(
() -> tracking.remove(sa)));

Future<?> f;
if (delayTime <= 0) { // (4)
f = exec.submit(sa);
} else
if (exec instanceof ScheduledExecutorService) { // (5)
f = ((ScheduledExecutorService)exec)
.schedule(sa, delayTime, unit);
} else {
f = genericScheduler.schedule(() -> { // (6)
Future<?> g = exec.submit(sa);
sa.add(Subscriptions.create( // (7)
() -> g.cancel(false)));
}, delayTime, unit);
}

sa.add(Subscriptions.create( // (8)
() -> f.cancel(false)));

return sa; // (9)
}
// ...

Our first truly complicated logic in this blog post works as follows:

  1. If the worker is unsubscribed, the method will return with a constant unsubscribed subscription. Note that schedule calls slipping past this will likely receive an unchecked RejectedExecutionException from the underlying thread pool. You can wrap the rest of the method with try-catch and return the same constant as necessary.
  2. We wrap the action into our ScheduledAction.
  3. Before the action gets even scheduled, we add it to the tracking structure and add callback that will remove the ScheduledAction from the tracking structure if it completes or gets unsubscribed directly. Note that due to idempotence, the remove() can't get into an infinite loop by calling unsubscribe() on the ScheduledAction again.
  4. In case there wasn't any delay time, we schedule the action directly and taking hold on the returned Future for it.
  5. In case the ExecutorService is also a ScheduledExecutorService, we can call schedule() on it with the delay parameters directly.
  6. Otherwise, we'll need some ScheduledExecutorService instance that will do the delaying for us, but we can't just schedule the action directly on it: it would run on the wrong thread. Instead, we need to schedule an intermediate task which when the time has come, will perform the actual scheduling, without delay, on the proper thread pool.
  7. We need to wire up the returned future to be able to cancel the tasks via a unsubscribe() call. Here we'll add the inner future to the ScheduledAction instance.
  8. Either scheduled directly or indirectly, we need a way to communicate the cancellation to the thread pool, therefore, we wrap the action of cancelling the future via cancel() into a Subscription and add it to the ScheduledAction. Here, you have the option to do an interrupting cancel or just a regular cancel. (RxJava does both depending on what thread issues the cancel call: if it is the same thread that runs the action, there is no need for interrupts.)
  9. The wrapper itself will be the cancellation token directly.
Because of the terminal-state-nature of the subscription-containers, even if (7) or (8) happens after the unsubscription of the ScheduledAction, they will get immediately cancelled. For extra eagerness, you can modify the ScheduledAction.run() to explicitly check isUnsubscribed() before calling the wrapped action.

The last tiny bit of code missing is an example for the genericScheduler. You can add a static final field to the worker and set it up like this:

    // ...
static final ScheduledExecutorService genericScheduler;
static {
genericScheduler = Executors.newScheduledThreadPool(1, r -> {
Thread t = new Thread(r, "GenericScheduler");
t.setDaemon(true);
return t;
});
}
// ...

Conclusion

In this post, I've shown how one can implement a wrapper for the actions to be scheduled and how it can be wired up properly to work with unsubscription at an individual task or at a worker level.

In the final part of this series, I'll talk about how to handle the case when the ExecutorService has multiple threads since we can't let non-delayed tasks reordered or run in parallel with each other.

Schedulers (part 3)

$
0
0

Introduction

In this blog post, I'm going to talk about the case when one wants to wrap an existing multi-threaded Executor and ensure the contracts on Scheduler and Worker are enforced in some manner.

The most important contract on Worker is that tasks submitted sequentially have to execute in the same order, but threads in the Executor would pick them up in random fashion and execute them out of order and in parallel.

The solution is to use a form of the queue-drain approach we saw many times and perform a trampolining of the scheduled actions. The queue will make sure the order is of the actions is kept and the drain logic will make sure only one drain action is running at a time so parallel execution is avoided.

The ExecutorScheduler

As  usual, we start with the skeleton of the class:

public final class ExecutorScheduler extends Scheduler {
final Executor exec;
public ExecutorScheduler(Executor exec) {
this.exec = exec;
}
@Override
public Worker createWorker() {
return new ExecutorWorker();
}

final class ExecutorWorker extends Worker
implements Runnable { // (1)
// data fields here
@Override
public Subscription schedule(
Action0 action) {
// implement
}
@Override
public Subscription schedule(
Action0 action, long delayTime,
TimeUnit unit) {
// implement
}
@Override
public void run() {
// implement
}
@Override
public boolean isUnsubscribed() {
// implement
}
@Override
public void unsubscribe() {
// implement
}
}
}

All worker instances will delegate the scheduling actions to the same underlying Executor. Interesting to note that if we make ExecutorWorker implement Runnable, we can save on the creation of a separate Runnable that is needed in the drain phase.

The queue-drain logic requires a queue and a work-in-progress indicator and the worker requires a tracking subscription container to support mass-unsubscription:

        // ...
final AtomicInteger wip = new AtomicInteger();
final Queue<ScheduledAction> queue = new ConcurrentLinkedQueue<>();
final CompositeSubscription tracking = new CompositeSubscription();
// ...

The use of ConcurrentLinkedQueue may trigger the performance enthusiasts (like me), since it would appear one only needs either an MpscLinkedQueue from JCTools, for example, or even my MpscLinkedArrayQueue only.

However, there is a tradeoff here: are we willing to pay the cost of cancelled task retention or not. This wasn't an issue with the standard Schedulers backed by ScheduledExecutorService instances because they removed cancelled task instances from their internal queue automatically (or a periodic purge does it in Java 6 environment). Such removal option is not available with JCTools or my queue, therefore, the best option, for now, is to use ConcurrentLinkedQueue when task retention is an issue. (One can, of course, implement a specialized queue and task object that know about each other so the unsubscription can locate the task itself inside the queue and CAS it out with a tombstone task.) Note however, that the removal will be an O(n) per task.

Given the base data fields, let's implement the easier methods of the ExecutorWorker:

        // ...
@Override
public boolean isUnsubscribed() {
return tracking.isUnsubscribed();
}
@Override
public void unsubscribe() {
queue.clear();
tracking.unsubscribe();
}
}
}

Note that since we are not in the control of the lifecycle of the Executor, it would be unwise to try and shut it down and even if we did it, other worker instances would stop working as well. The best we can do is to track the tasks submitted to this particular worker and mass-unsubscribe only them.

Now come the complicated parts. First, let's see the non-delayed schedule() method:

    @Override
public Subscription schedule(Action0 action) {
if (isUnsubscribed()) {
return Subscriptions.unsubscribed();
}
ScheduledAction sa =
new ScheduledAction(action);
tracking.add(sa);
sa.add(Subscriptions.create(
() -> tracking.remove(sa))); // (1)

queue.offer(sa); // (2)

sa.add(Subscriptions.create(
() -> queue.remove(sa))); // (3)

if (wip.getAndIncrement() == 0) { // (4)
exec.execute(this); // (5)
}

return sa;
}

The method can't delegate to the delayed overload anymore and needs its own logic:

  1. We create our ScheduledAction (from part 2) and add the logic to remove itself from the tracking structure on unsubscription.
  2. We offer the action to the queue which will keep the FIFO order of the actions submitted sequentially.
  3. Then we add the logic to remove our task from the queue in case it gets unsubscribed. Note that this step is O(n) where n is the tasks waiting in the queue before the instance.
  4. We should allow only a single drainer which can only happen if the wip counter transitions from 0 to 1.
  5. If the current call to schedule won the right to drain the queue, we submit the worker itself so its run() method can poll the queue for available tasks.
Note that even if we got an ExecutorService, we can't associate the Future of submitting this with any particular action, therefore, cancellation has to happen some indirect way.

Now let's continue with the run() method's implementation:


    @Override
public void run() {
do {
if (isUnsubscribed()) { // (1)
queue.clear();
return;
}
ScheduledAction sa = queue.poll(); // (2)
if (sa != null && !sa.isUnsubscribed()) {
sa.run(); // (3)
}
} while (wip.decrementAndGet() > 0); // (4)
}

A relatively straightforward drain logic:

  1. We check if the worker has been unsubscribed in the meantime, if so, we clear the queue (just in case) and return.
  2. We poll the next task from the queue.
  3. Since a removal or unsubscription could clear a queue while run() is executing, we need to check for null and we need to check if the particular ScheduledAction has been unsubscribed or not. If not, we execute it.
  4. We decrement the wip counter until it reaches zero, where it becomes safe to re-schedule the method on the Executor with more tasks to drain if necessary.
Finally, the most complicated method is the delayed schedule():

    @Override
public Subscription schedule(
Action0 action,
long delayTime,
TimeUnit unit) {

if (delayTime <= 0) {
return schedule(action); // (1)
}
if (isUnsubscribed()) {
return Subscriptions.unsubscribed(); // (2)
}

ScheduledAction sa =
new ScheduledAction(action);
tracking.add(sa);
sa.add(Subscriptions.create(
() -> tracking.remove(sa))); // (3)

ScheduledExecutorService schedex;
if (exec instanceof ScheduledExecutorService) {
schedex = (ScheduledExecutorService) exec; // (4)
} else {
schedex = CustomWorker.genericScheduler; // (5)
}

Future<?> f = schedex.schedule(() -> { // (6)

queue.offer(sa); // (7)

sa.add(Subscriptions.create(
() -> queue.remove(sa)));

if (wip.getAndIncrement() == 0) {
exec.execute(this);
}

}, delayTime, unit);

sa.add(Subscriptions.create(
() -> f.cancel(false))); // (8)

return sa;
}

It looks very similar to the non-delayed schedule() method, but it has to handle the delay in some fashion:

  1. To avoid the unnecessary overhead, we delegate scheduling of tasks with non-positive delay back to the non-delayed schedule() method.
  2. We return an unsubscribed Subscription if the worker has been unsubscribed.
  3. We wrap the action with our ScheduledAction, add it to the tracking structure and register a removal action as well.
  4. We need a scheduling-capable service for the delayed execution, so we check whether the Executor we've got is capable of doing it itself,
  5. or we need to arrange it ourselves, for example, by using the genericScheduler of our CustomWorker from part 2.
  6. With a scheduling service in hand, we schedule a task that will then enqueue the actual task after the specified delay.
  7. In this helper task, we perform the same steps as in the non-delayed schedule() call, but we don't need to wrap our ScheduledAction again. It offers the ScheduledAction to the queue, creates the remove action and does the wip-increment trick to jump-start the drain if necessary.
  8. Once the delayed task returns a Future, we add a cancellation action to the ScheduledAction so the unsubscription will cancel it.
Naturally, we will try the ExecutorScheduler out:

ExecutorService exec = Executors.newFixedThreadPool(3);
try {
Scheduler s = new ExecutorScheduler(exec);

Observable<Integer> source = Observable.just(1)
.delay(500, TimeUnit.MILLISECONDS, s)
.doOnNext(v -> {
try {
Thread.sleep(1000);
} catch (InterruptedException ex) {
ex.printStackTrace();
}
System.out.println(Thread.currentThread());
});

TestSubscriber<Integer> ts1 = new TestSubscriber<>();
TestSubscriber<Integer> ts2 = new TestSubscriber<>();
TestSubscriber<Integer> ts3 = new TestSubscriber<>();

source.subscribe(ts1);
source.subscribe(ts2);
source.subscribe(ts3);

ts1.awaitTerminalEvent();
ts1.assertNoErrors();
ts1.assertValue(1);

ts2.awaitTerminalEvent();
ts2.assertNoErrors();
ts2.assertValue(1);

ts3.awaitTerminalEvent();
ts3.assertNoErrors();
ts3.assertValue(1);
} finally {
exec.shutdown();
}

Which prints something like this:

Thread[pool-1-thread-3,5,main]
Thread[pool-1-thread-1,5,main]
Thread[pool-1-thread-2,5,main]

Conclusion

In this part about Schedulers, I've talked about the need to serialize execution of non-delayed actions scheduled by the same worker in case the actual underlying Executor is multi-threaded and can't guarantee it.

In the next post, I'll talk about the case when some scheduling framework, such as a GUI event loop, which don't offer a Future-like cancellation possibility and I'll show how one can interoperate with such API through a Scheduler.

Schedulers (part 4 - final)

$
0
0

Introduction

In this final blog post about Schedulers, I'll talk a bit about services which don't expose any ExecutorService facilities, such as when one needs to interact with GUI event loops.

Working with non-Executor-based services

Some scheduling frameworks, such as Java's own AWT event loop doesn't offer Future-like scheduling capabilities, but only a single method to execute a task on them. Other frameworks may offer a pair of add/remove methods for tasks.

Let's assume we have the following API available to perform task-scheduling and cancellation on some GUI event loop:

public interface GuiEventLoop {
void run(Runnable task);
void cancel(Runnable task);
}

public static final class EDTEventLoop
implements GuiEventLoop {
@Override
public void run(Runnable task) {
SwingUtilities.invokeLater(task);
}

@Override
public void cancel(Runnable task) {
// not supported
}
}

Here, I defined an example API and an implementation that wraps the Java AWT Event Dispatch Thread. Unfortunately, EDT doesn't offer any way of cancelling already submitted tasks, but since such tasks are not supposed to take too long anyway, that's usually not a shortcoming in applications.

Naturally, one can wrap convert the above invocation direclty into an Executor:

Executor exec = SwingUtilities::invokeLater;

and use the ExecutorScheduler from part 3, but it'd, usually, add unnecessary overhead and I'd also like to show how to deal with the case when a task-removal and cancellation is available through a specific method in the GUI framework.

Since GUI event loops are single-threaded, we don't need to worry about serialization and trampolinig in our Worker implementation and we can start out with a simpler skeleton for our GuiScheduler:

public final class GuiScheduler extends Scheduler {

final GuiEventLoop eventLoop;

public GuiScheduler(GuiEventLoop el) {
this.eventLoop = el;
}

@Override
public Worker createWorker() {
return new GuiWorker();
}

final class GuiWorker extends Worker {
final CompositeSubscription tracking =
new CompositeSubscription();
@Override
public void unsubscribe() {
tracking.unsubscribe();
}

@Override
public boolean isUnsubscribed() {
return tracking.isUnsubscribed();
}

@Override
public Subscription schedule(Action0 action) {
// implement
}

@Override
public Subscription schedule(
Action0 action,
long delayTime,
TimeUnit unit) {
// implement
}
}
}

There isn't anything special yet: we are going to delegate to the same GuiEventLoop instance and we track tasks scheduled by individual GuiWorker instances separately. Since we expect the GuiEventLoop to be single-threaded, there is no need to do queue-drain here and thus have the worker extend Runnable. Let's see the non-delayed schedule() implementation first:

    @Override
public Subscription schedule(Action0 action) {
if (isUnsubscribed()) { // (1)
return Subscriptions.unsubscribed();
}
ScheduledAction sa = new ScheduledAction(action);
tracking.add(sa);
sa.add(Subscriptions.create(
() -> tracking.remove(sa))); // (2)

Runnable r = () -> { // (3)
if (!sa.isUnsubscribed()) {
sa.run();
}
};

eventLoop.run(r); // (4)

sa.add(Subscriptions.create(
() -> eventLoop.cancel(r))); // (5)

return sa;
}


It contains quite a few familiar steps:

  1. In case the worker has been unsubscribed, we just return an unsubscribed Subscription instance.
  2. We wrap the action with our ScheduledAction and hookup the tracking and removal logic.
  3. In this example, we care about that if the ScheduledAction is unsubscribed, we eagerly won't execute its body. Because the eventLoop API expects the same Runnable instance for cancellation and ScheduledAction.run() doesn't do an isUnsubscribed() check on itself, we need to wrap a small logic into a Runnable.
  4. We submit this wrapper runnable to the eventLoop API,
  5. then add a unsubscription action which will remove it as necessary. Note if we'd done this the other way around and the worker itself was unsubscribed just before (4), we'd immediately call cancel with an r that is not in the event loop and then schedule r regardless and have a larger retention window than in the current case.
Since our target API doesn't offer any delayed scheduling capabilities (that comes in handy with periodic tasks such as animations), we have to rely on the genericScheduler again from part 2:

    @Override
public Subscription schedule(
Action0 action,
long delayTime,
TimeUnit unit) {

if (delayTime <= 0) { // (1)
return schedule(action);
}
if (isUnsubscribed()) { // (2)
return Subscriptions.unsubscribed();
}
ScheduledAction sa =
new ScheduledAction(action);
tracking.add(sa);
sa.add(Subscriptions.create(
() -> tracking.remove(sa))); // (3)

Future<?> f = CustomWorker.genericScheduler // (4)
.schedule(() -> {
Runnable r = () -> {
if (!sa.isUnsubscribed()) {
sa.run();
}
};

eventLoop.run(r);

sa.add(Subscriptions.create(
() -> eventLoop.cancel(r))); // (5)

}, delayTime, unit);

sa.add(Subscriptions.create(
() -> f.cancel(false)));

return sa;
}

Most of the time, the algorithm boils down to the same underlying structure with lots of similarities:

  1. We treat any non-positive delay as a regular schedule() call.
  2. We return if the worker itself was already unsubscribed.
  3. We perform the usual wrapping and hookups.
  4. We take our genericScheduler and schedule a delayed action that will then relay the real action to the eventLoop,
  5. which happens the same way as with the regular schedule() call: we wrap the action into a runnable that checks for its unsubscribed state, submit it to the event loop and add a cancel action.
Finally, let's use it:

Scheduler s = new GuiScheduler(new EDTEventLoop());

Observable source = Observable.just(1)
.delay(500, TimeUnit.MILLISECONDS, s)
.doOnNext(v -> {
try {
Thread.sleep(1000);
} catch (InterruptedException ex) {
ex.printStackTrace();
}
System.out.println(Thread.currentThread());
});

TestSubscriber ts1 = new TestSubscriber<>();
TestSubscriber ts2 = new TestSubscriber<>();
TestSubscriber ts3 = new TestSubscriber<>();

source.subscribe(ts1);
source.subscribe(ts2);
source.subscribe(ts3);

ts1.awaitTerminalEvent();
ts1.assertNoErrors();
ts1.assertValue(1);

ts2.awaitTerminalEvent();
ts2.assertNoErrors();
ts2.assertValue(1);

ts3.awaitTerminalEvent();
ts3.assertNoErrors();
ts3.assertValue(1);


Which should print:

Thread[AWT-EventQueue-0,6,main]
Thread[AWT-EventQueue-0,6,main]
Thread[AWT-EventQueue-0,6,main]

Conclusion

In this final blog post about Schedulers, I've shown how one can wrap some event-loop based framework's API and convert it to RxJava's Scheduler API.

Generally though, there are many subtleties possible with the Scheduler API or with APIs we'd like to wrap with it. Such 'individual' cases are hard to generalize upfront in blog posts, so if you have some interesting or difficult API to wrap, try your luck in the rx-java topic on StackOverflow; our RxJava google group or you can contact me more directly in the comments or on twitter (@akarnokd).

Reactive-streams seems to have become more widely known lately, but since it doesn't offer much beyond a set of interoperation interfaces (i.e., no flatMap etc.), many started to write their own one-time Publishers over it and got confused about how its Subscription model should behave. Since RxJava 2.0 will support reactive-streams API natively anyway, luckily, our knowledge about Producers will come just in handy when doing reactive-streams-like Subscriptions. In the next series of blog post, I'll talk about the reactive-streams API and how one can convert an RxJava Producer into such a Subscription.

The Reactive-Streams API (part 1)

$
0
0

Introduction


The Reactive-Streams API, heavily influenced by RxJava, is (in my opinion) the most beautiful API describing (a)synchronous, cancellable and backpressure-able dataflows to date, replacing the original and limited IObservable/IObserver approach invented by Erik Meijer.

Back in April, I spent a week and tried porting RxJava 1.x over this new API. At first, the biggest hurdle was the lack of resource-management of the new Subscriber interface and how RxJava's Producer and Subscription interfaces were merged into a single Subscription interface. However, once I started implementing various infrastructure supporting classes, I started appreciating the new design, and perhaps the most important outcome is how the entire backpressure awareness just clicked into the right place in my mind.

With this new insight, I started looking at RxJava's operators and Producer implementations and I was suddenly able to discover bugs and optimization possibilities quite easily, not to mention, start a blog series about how things (should) work in RxJava.

In this blog series, I'm going to talk about this new reactive-streams API and its relation to current RxJava 1.x constructs.

Package and naming confusions

Knowing RxJava in and out, perhaps the first stumbling block about reactive-streams (RS) is how concepts are named differently.

rx.Observable is called org.reactivestreams.Publisherand is now an interface. Since Java doesn't and probably won't support extension methods, having a Publisher interface returned as a source doesn't really help with composing operators over it. Luckily, the current RxJava fluent approach can still remain as is by extending this new interface.

Unsubscription through the rx.Subscription and backpressure through the rx.Producer interfaces are now somewhat unified into a single interface, named org.reactivestreams.Subscription. The new interface uses cancel() instead of unsubscribe() and doesn't have (nor really needs) the means to check for a isUnsubscribed() state. Since RS doesn't specify any means of resource management, one will need to constantly wrap the new cancel() method into a resource management instance to be able to use it with the containers. Due to this name confusion, RxJava 2.0 will most likely change the resource management classes and interfaces into the C#-esque Disposables.

There is no simple rx.Observer equivalent, but the rx.Subscriber looks somewhat similar to org.reactivestreams.Subscriber. Instead of it being unsubscribable directly, the new Subscriber receives a 'control' object that lets it cancel the connection and request more values. In addition, instead of having two methods, onStart() and setProducer() - which pose quite a confusion about how one should jump-start a Subscriber in RxJava -, there is only a single onSubscribe() method taking an RS Subscription. The former calls to the protected rx.Subscriber.request(n) now have to store this RS Subscription instance and call rs.Subscription.request(n) on it.

There were some debates about how the completion signal's method should be called and was settled on onComplete() instead of RxJava's and Rx.NET's onCompleted() name. Hopefully, our IDE's auto-complete feature will save us in this regard.

Finally, RS specifies a combined interface of rs.Publisher and rs.Subscriber by the name of Processor, which resembles to the Subjects API inside RxJava. Since this Processor is an interface, the only change to rx.Subject will be to implement it.


Java 9+ Flow

There are some clashes between RxJava and RS, but you might have heard Doug Lea wants to include the reactive-streams idiom in Java 9 under container object java.util.concurrent.Flow. The naming matches RS but the packages are different. I was and I'm still skeptical about this move:
  • This "common platform" won't be that common: only Java 9+ users will benefit from it. Having RS, which is Java 6+ and thus Android-friendly, is a better option; most projects already include many other libraries and plus 2 (RS + RxJava 2.0) is not an issue.
  • Java 9 won't use it internally at all. No new async NIO, networking, File I/O and UI events based on the Observer++ pattern seems to be planned. Not to mention, without fluent API support, most users can't go zero-dependency with it. The situation can be remedied if Flow is backported to previos Java JDKs (with the same package location) in an update. Still, without extension methods in Java, users have to always rely on extra libraries and instance wrappings.
  • I'm not sure the whole reactive paradigm is final in terms of capturing properties: for example, Applied Duality's AsyncObservable capturing the latency of onXXX methods might be some day incorporated in RS 2.0 (although I don't really see the need for it). Swapping out the dependencies to RS 2.0 and RxJava 3.0 might be much simpler than changing JDK's Flow API. On a historical note, Rx.NET got its IObservable/IObserver included in the BCL, and I think they painted themselves into a corner and in order them to join the RS-paradigm, they now have to figure out a way to have the new set of interface structures live alongside the old one, similar to how pre-generics and post-generics collections ended up. Note that I assume here they accept RS as the new basis for Rx.NET 3.0 and don't try to do something more complicated.
 

     Critique of RS

    I'm completely satisfied with the four new interfaces of RS, but less so with the textual specification about them. I believe it is over-restrictive and prevents some sensible API implementations.

    No exceptions other than NullPointerException

    Although methods are allowed to throw NPE if parameters of various methods are null, however, the components can end up in invalid state or the Subscription.request() could be called with an invalid non-positive value. The current RS 1.0 specification doesn't allow handing these cases the way most Java APIs do by throwing IllegalArgumentException or IllegalStateException.

    One can argue that such exceptions should be sent through the onError() method, but I'll show potential situation that makes this impossible if adhering to the specification.

    Let's assume one has a Subscriber and accidentally subscribes it to two different Publishers. By definition, this is not allowed and the onSubscribe() method should reject a second Subscription:

        // ...
    Subscription subscription;
    @Override
    public void onSubscribe(Subscription s) {
    if (subscription != null) {
    s.cancel();
    onError(new IllegalStateException("§x.y: ..."));
    return;
    }
    this.subscription = s;
    s.request(Long.MAX_VALUE);
    }

    This may appear to work, but the moment the first Publisher goes async, two problems arise:
    • The second onSubscribe() call's onError() is now, potentially, concurrently invoked with the first Publisher's onNext() call, which is correctly forbidden by the specification.
    • Storing the first subscription can't be plain and requires an AtomicReference in case there is a subscription race due to multiple uses, adding overhead to a Subscriber implementation.
    Therefore, the only safe way to implement onSubscribe() could be something like this:

        // ...
    final AtomicReference subscription = ...;
    @Override
    public void onSubscribe(Subscription s) {
    if (!subscription.compareAndSet(null, s)) {
    s.cancel();
    new IllegalStateException("§x.y: ...").printStackTrace();
    return;
    }
    s.request(Long.MAX_VALUE);
    }

    and hope the standard out or log is monitored pr one needs to serialize out the subscriber's onXXX() methods all the time.

    Instead of this, I'd simply throw the IllegalArgumentException and make the subscribe() fail.

    Illegal request amount

    Requests through the Subscriptions should be positive, otherwise, the subscriber should be notified via onError().

        // ...
    @Override
    public void request(long n) {
    if (n <= 0) {
    subscriber.onError(new IllegalArgumentException("§x.y: ..."));
    return;
    }
    // ...
    }


    Again, the problem comes when request() is invoked asynchronously, for example, due to an observeOn() trying to replenish its input queue while the Subscription is still producing values through onNext(). The onError() is now racing with onNext() again, which is forbidden.
    The resolution and my suggestions are similar to the IllegalStateException: one can print/log the error or instead throw it back at the caller.


    The request() can't throw NPE

    Even though various onXXX() methods can at least throw a NullPointerException, the Subscription.request() can't. One has to try and bounce such exception back to the Subscriber and if that fails as well, there is nowhere it could go but to standard out or log.


    Processors require Subscriptions

    The RS is heavily biased towards cold observable sequences where having a Subscription is a correct interoperation element. However, hot Observables, such as RxJava Subjects may or may not receive Subscriptions before their onNext() is called.

    For example, if one uses a PublishSubject to multicast an Observable by calling Observable.subscribe(PublishSubject), the PublishSubject will receive a Subscription. However, if one uses PublishSubject to multicast mouse movement events, there is no way it can be cancelled (or backpressured) and thus having a Subscription there is unnecessary.

    Therefore, one either has to use some dummy Subscription or, as I see, make the call to onSubscribe() optional for Processors.


    Processors have to support backpressure

    Backpressure support might be impossible and infeasible with Processors: mouse moves can't be backpressured, therefore, a Subject would need to buffer and/or drop overflown values. Implementing it for AsyncSubject is trivial, but other Subject types have to be turned into some kind of hybrid ReplaySubject. (Naturally, PublishSubject can by default drop unrequested values for its subscribers as if they weren't there, but the continuous value delivery guarantee of Subjects is too valuable to be subverted in my opinion.)

    A conforming implementation would impose quite an overhead for all subscribers by preventing 'write-through' and potentially lock-stepping them and slowing the processing down to the slowest requestor.

    Instead, I'd weaken the related specification points and allow optional backpressure support for Processors.


    Subscriptions will be conservative regardless

    The specification suggests that Subscriptions can be implemented without thinking about concurrency too often. If you remember about the requirements of Producers in RxJava, I mentioned the requirement of thread-safety and reentrant-safety.

    Reentrant-safety will still be critical part of the Subscription.request() implementation to avoid unnecessary recursion and to handle the case when the onNext() calls request() before it processes the current value which would trigger another (potentially) recursive call to onNext() ad infinitum.

    The need for thread-safety can be 'proven' by the behavior of the following chain. Let's assume a chain is subscribed on the main thread, outside any RxJava schedulers and starts producing values through an observeOn() operator. The consumer of the sequence is then periodically requesting more data which request() call is forwarded to the source still emitting from the main thread. At this point, two threads are in or enter the request() method and without proper atomics/synchronization, the reentrancy check and proper request accounting wouldn't work. Now one would think the observeOn() could serialize access to its Subscription (the main source) but since the source runs outside the Schedulers, such serialization would be no-op from the downstream's perspective and ineffective from the upstream's perspective.

    Since the source Subscriber can't possibly know the call pattern to its request() method, it has to be conservative and use one of the approaches I blogged about in respect to Producers. Naturally, this also implies that the cancel() method has to touch a volatile variable in any implementations.

    Conclusion

    These concerns didn't just arise while writing this post, but I've already tried to notify the RS designers about them. Unfortunately, my concerns didn't got through and my code didn't win arguments. We could argue about my issues endlessly, but instead, I'd rather write code and prove my points there.

    Regardless, I find RS to be a good basis to build upon with some slight phylosophical adjustments:
    • Error handling: to paraphrase the latest Godzilla movie: "Let them throw!".
    • Processors have the option to do backpressure and cancellation on their Subscriber side.
    • Most non-trivial Subscriptions have to be conservative and made threadsafe.
    In the next post, I'll start with the two most basic Producers of RxJava, namely the SingleProducer and SingleDelayedProducer and show how they can be made into an RS SingleSubscription and SingleDelayedSubscription.

    The Reactive-Streams API (part 2)

    $
    0
    0

    Introduction

    In this blog post, I'm going to take our SingleProducer and SingleDelayedProducer classes and convert them into a reactive-streams based Subscriptions.

    At first, one might think the conversion is going to be troublesome, but luckily, if you can already thing in how you'd implement the request() method on a rx.Producer, you are 75% there. The final 25% comes from the idea how you'd move the rx.Subscriber.isUnsubscribed() logic into request() since the rs.Subscriber doesn't extend rx.Subscription (nor any other resource-management interface).

    The SingleSubscription

    Since the SingleSubscription itself isn't that complicated, I'm going to show it in one go:

    import org.reactivestreams.*;

    public final class SingleSubscription<T>
    extends AtomicBoolean implements Subscription {
    private static final long serialVersionUID = 1L;

    final T value; // (1)
    final Subscriber<? super T> child;
    volatile boolean cancelled; // (2)

    public SingleSubscription(T value,
    Subscriber<? super T> child) { // (3)
    this.value = Objects.requireNonNull(value);
    this.child = Objects.requireNonNull(child);
    }
    @Override
    public void request(long n) {
    if (n <= 0) {
    throw new IllegalArgumentException(
    "n > 0 required"); // (4)
    }
    if (compareAndSet(false, true)) {
    if (!cancelled) { // (5)
    child.onNext(value);
    if (!cancelled) {
    child.onComplete();
    }
    }
    }
    }
    @Override
    public void cancel() {
    cancelled = true; // (6)
    }
    }

    That's it! Wait, that's it? Yes, it is no accident that I've been showing Producer implementations so far that can be transformed into a reactive-streams Subscription with relatively little effort. But still, here are the explanation for the major properties of this new implementation:

    1. We have an instance field for the constant value and the target Subscriber as before,
    2. however, since isUnsubscribed() is not part of the RS Subscriber and unsubscription comes in the form of a cancel() call, we need to store the cancelled state ourselves, in a volatile field. If you recall, I mentioned that you can't be sure by what and when request() (or in fact, cancel()) will be called, therefore, one needs to make sure things are thread-safe.
    3. Since RS doesn't like null values, we capture them early in the constructor.
    4. My "Let them throw!" philosophy dictates that non-positive requests are programming errors which should yield a nice IllegalArgumentException.
    5. Because there is no child.isUnsubscribed() method anymore, we check the volatilecancelled variable everywhere instead.
    6. Our idempotent cancel just sets the cancelled flag atomically.

    The SingleDelayedSubscription

    Given the simplicity of SingleSubscription, how hard could it be to convert SingleDelayedProducer?

    public final class SingleDelayedSubscription<T> 
    extends AtomicInteger implements Subscription {
    /** */
    private static final long serialVersionUID = -1L;

    T value;
    final Subscriber<? super T> child;

    static final int CANCELLED = -1; // (1)
    static final int NO_VALUE_NO_REQUEST = 0;
    static final int NO_VALUE_HAS_REQUEST = 1;
    static final int HAS_VALUE_NO_REQUEST = 2;
    static final int HAS_VALUE_HAS_REQUEST = 3;

    public SingleDelayedSubscription(Subscriber<? super T> child) {
    this.child = Objects.requireNonNull(child);
    }
    @Override
    public void request(long n) {
    if (n <= 0) {
    throw new IllegalArgumentException("n > 0 required");
    }
    for (;;) {
    int s = get();
    if (s == NO_VALUE_HAS_REQUEST
    || s == HAS_VALUE_HAS_REQUEST
    || s == CANCELLED) { // (2)
    return;
    }
    if (s == NO_VALUE_NO_REQUEST) {
    if (!compareAndSet(s, NO_VALUE_HAS_REQUEST)) {
    continue;
    }
    }
    if (s == HAS_VALUE_NO_REQUEST) {
    if (compareAndSet(s, HAS_VALUE_HAS_REQUEST)) {
    T v = value;
    value = null;
    child.onNext(v);
    if (get() != CANCELLED) { // (3)
    child.onComplete();
    }
    }
    }
    return;
    }
    }

    public void setValue(T value) {
    Objects.requireNonNull(value);
    for (;;) {
    int s = get();
    if (s == HAS_VALUE_NO_REQUEST
    || s == HAS_VALUE_HAS_REQUEST
    || s == CANCELLED) { // (4)
    return;
    } else
    if (s == NO_VALUE_NO_REQUEST) {
    this.value = value;
    if (!compareAndSet(s, HAS_VALUE_NO_REQUEST)) {
    continue;
    }
    } else
    if (s == NO_VALUE_HAS_REQUEST) {
    if (compareAndSet(s, HAS_VALUE_HAS_REQUEST)) {
    child.onNext(value);
    if (get() != CANCELLED) { // (5)
    child.onComplete();
    }
    }
    }
    return;
    }
    }

    @Override
    public void cancel() {
    int state = get();
    if (state != CANCELLED) { // (6)
    state = getAndSet(CANCELLED);
    if (state != CANCELLED) {
    value = null;
    }
    }
    }
    }

    Looks quite similar to the original state-machine, but it has an additional CANCELLED state (1..6), which is atomically swapped in. We don't really need to check for this state before onNext() because the preceding compareAndSet() would fail anyway, but we can check it just before calling onComplete().

    Why don't we use a volatile cancelled flag instead of this new state? You could naturally do that and the resulting Subscription would be equally correct. It is a matter of personal preference: you can add an extra instance field or extend the state machine to include a cancelled state. The primary reason here, mostly, is to show an example of this latter alternative.

    The RangeSubscription

    I'm not going to convert all previous Producers into Subscriptions here, but I'd like to show a second example for including a cancelled state in the state machine.

    public final class RangeSubscription 
    extends AtomicLong implements Subscription {
    /** */
    private static final long serialVersionUID = 1L;

    final Subscriber<? super Integer> child;
    int index;
    final int max;

    static final long CANCELLED = Long.MIN_VALUE; // (1)

    public RangeSubscription(
    Subscriber<? super Integer> child,
    int start, int count) {
    this.child = Objects.requireNonNull(child);
    this.index = start;
    this.max = start + count;
    }
    @Override
    public void request(long n) {
    if (n <= 0) {
    throw new IllegalArgumentException(
    "n > required");
    }
    long r;
    for (;;) {
    r = get();
    if (r == CANCELLED) { // (2)
    return;
    }
    long u = r + n;
    if (u < 0) {
    u = Long.MAX_VALUE;
    }
    if (compareAndSet(r, u)) {
    break;
    }
    }
    if (r != 0L) {
    return;
    }
    for (;;) {
    r = get();
    if (r == CANCELLED) { // (3)
    return;
    }
    int i = index;
    int m = max;
    long e = 0;
    while (r > 0L && i < m) {
    child.onNext(i);
    if (get() == CANCELLED) { // (4)
    return;
    }
    i++;
    if (i == m) {
    child.onComplete();
    return;
    }
    r--;
    i++;
    e++;
    }
    index = i;
    if (e != 0) {
    for (;;) {
    r = get();
    if (r == CANCELLED) { // (5)
    return;
    }
    long u = r - e;
    if (u < 0) {
    throw new IllegalStateException(
    "more produced than requested!");
    }
    if (compareAndSet(r, u)) {
    break;
    }
    }
    }
    if (r <= 0L) {
    break;
    }
    }
    }
    @Override
    public void cancel() {
    if (get() != CANCELLED) { // (6)
    getAndSet(CANCELLED);
    }
    }
    }

    For brevity, I've omitted the fast-path logic here. The rest is, again, similar to the original RangeProducer structure, but now that the cancellation state is merged into the requested accounting, we need to re-read the current requested amount and check for a CANCELLED value (1) almost everywhere (2..5). Note that the emission accounting can't be a simple getAndAdd() anymore, because even if CANCELLED would be -1, one could, in theory, emit Long.MAX_VALUE - 1 and wrap the counter, losing the cancelled state information. Again, using getAndSet() to swap in the terminal state atomically and in idempotent fashion (6).


    Conclusion

    In this part, I've shown two approaches to convert from a rx.Producer into an RS Subscription and keep the unsubscription behavior intact. Naturally, they involve tradeoffs: instance size if using a separate cancellation flag or algorithm complexity if cancellation is woven into the state machine.

    In the next part, I'm going to show how one can deal with the loss of another rx.Subscriber functionality: the add(rx.Subscriber) option to associate resources with the downstream Subscriber.

    The Reactive-Streams API (part 3)

    $
    0
    0

    Introduction


    In this episode, I'm going to talk about how the rx.Subscriber's ability to register resources with it can be converted into the Reactive-Streams API. However, since RS doesn't specify any means of resource management, we need to introduce (rename the rx.Subscription) containers ourselves and attach them to the RS Subscriber's cancellation logic in some way.

    Subscription vs. Subscription

    To avoid confusion, RxJava 2.0 will replace the XXXSubscriptions with XXXDisposables classes and interfaces, however, I'm not going to detail these classes here, but only show the new base interfaces for resource management:

    interface Disposable {
    boolean isDisposed();
    void dispose();
    }

    interface DisposableCollection extends Disposable {
    boolean add(Disposable resource);
    boolean remove(Disposable resource);
    boolean removeSilently(Disposable resource);

    void clear();
    boolean hasDisposables();
    boolean contains(Disposable resource);
    }


    The usage rules will remain the same: thread safety and idempotence.


    The DisposableSubscription

    The most basic way of adding resource management is to wrap around an existing Subscription with one that hijacks the cancel() method and calls dispose() on the underlying composite:

    public final class DisposableSubscription
    implements Disposable, Subscription {
    final Subscription actual;
    final DisposableCollection collection;
    public DisposableSubscription(
    Subscription actual,
    DisposableCollection collection) {
    this.actual = Objects.requireNonNull(actual);
    this.collection = Objects.requireNonNull(collection);
    }
    public boolean add(Disposable resource) {
    return collection.add(resource);
    }
    public boolean remove(Disposable resource) {
    return collection.remove(resource);
    }
    public boolean removeSilently(Disposable resource) {
    return collection.remove(resource);
    }
    @Override
    public boolean isDisposed() {
    return collection.isDisposed();
    }
    @Override
    public void dispose() {
    cancel();
    }

    @Override
    public void cancel() {
    collection.dispose();
    actual.cancel();
    }
    @Override
    public void request(long n) {
    actual.request(n);
    }
    }

    By implementing the Disposable interface, the DisposableSubscription itself can now be added to a disposable-container and participate in a complex dispose network. However, most of the time you'd want to avoid extra allocation, therefore, the structure above will be likely inlined into some other class, perhaps into the Subscriber participating in a lift() call.

    If you remember the RxJava specification and pitfall #2, you shouldn't unsubscribe your downstream, because it is likely you'll trigger a premature release of its resources.

    (This is currently somewhat an issue with RxAndroid's LifecycleObservable where one can attach a takeUntil()-like operator in the middle of a chain which instead of sending out onCompleted(), it unsubscribes its downstream subscriber.)

    With RS, such downstream unsubscription is practically infeasible. Each level can only either pass the Subscription as-is (and thus can't attach resources to it) or wrap it with a DisposableSubscription-like class and forward that downstream as just a Subscription. If you call the cancel() on your level, it can't call a cancel() of a class that wraps your Subscription in any way.

    As always, you still can shoot yourself in the foot, but it takes much more effort than in RxJava today and the rule won't change: you shouldn't try to cancel/dispose your downstream's resources nor share them across the chain of operators.


    TakeUntil

    Now let's see how one can implement the takeUntil() operator with such externalized resource management (split into code parts for better readability):

    public final class OperatorTakeUntil<T> 
    implements Operator<T, T> {
    final Publisher<?> other;
    public OperatorTakeUntil(Publisher<?> other) {
    this.other = Objects.requireNonNull(other);
    }
    @Override
    public Subscriber<? super T> call(
    Subscriber<? super T> child) {
    Subscriber<T> serial =
    new SerializedSubscriber<>(child);

    SubscriptionArbiter arbiter =
    new SubscriptionArbiter(); // (1)
    serial.onSubscribe(arbiter);

    SerialDisposable sdUntil = new SerialDisposable(); // (2)
    SerialDisposable sdParent = new SerialDisposable(); // (3)

    So far, it looks similar to the current RxJava implementation: we wrap the child to serialize out the potentially asynchronous onError() or onCompleted() emitted by the other Publisher.

    We create a SubscriptionArbiter (a conversion of ProducerArbiter) for the following reason: let's assume, we are still in the call() method when the other source, already subscribed to, emits a notification of any kind. We'd like to forward it to the child Subscriber, however, unless it has a Subscription available, we can't call onXXX methods on it just yet. Such a 'real' subscription through the main path will only arrive once the operator-chain starts to call onSubscribe(). I'll talk about this in more detail in the next blog post.

    However, since the way the cancellation option is presented to Subscribers, we need a way (2) to pass the Subscription from the other Subscriber back to the parent Subscriber so if either of them reaches a terminal state, it can cancel() the other one. Since the other may get its Subscription and cancellation before the parent does, we need to be prepared to cancel the parent's subscription on reception as well (3).

                // ...
    Subscriber<T> parent = new Subscriber<T>() {
    DisposableSubscription dsub;
    @Override
    public void onSubscribe(Subscription s) {
    DisposableSubscription dsub =
    new DisposableSubscription(s,
    new DisposableList()); // (1)
    dsub.add(sdUntil); // (2)
    sdParent.set(dsub);
    arbiter.setSubscription(dsub); // (3)
    }
    @Override
    public void onNext(T t) {
    serial.onNext(t);
    }
    @Override
    public void onError(Throwable t) {
    serial.onError(t);
    sdParent.cancel(); // (4)
    }
    @Override
    public void onComplete() {
    serial.onComplete();
    sdParent.cancel();
    }
    };


    The parent Subscriber is a slightly different: we need to handle the incoming Subscription and wire up the cancellation network:

    1. We create a DisposableSubscription wrapper with a List-based underlying collection.
    2. We add the SerialDisposable - that will hold the Disposable to the other Subscription - to this composite. We also add the DisposableSubscription to the sdParent to allow the other Subscriber to finish before the parent one can even start.
    3. We hand the wrapper down to the serializer.
    4. In case an error or completion event happens, we'll make sure the composite is cancelled/disposed on the spot. Since the composite holds a reference to the other Subscriber's subscription, this will also cancel that stream.
    As the final segment, we need to create a Subscriber to the other stream and ensure it completes the parent:

            // ...
    Subscriber<Object> until = new Subscriber<Object>() {
    @Override
    public void onSubscribe(Subscription s) {
    sdUntil.set(Disposables.create(s::cancel)); // (1)
    s.request(Long.MAX_VALUE);
    }
    @Override
    public void onNext(Object t) {
    parent.onComplete(); // (2)
    }
    @Override
    public void onError(Throwable t) {
    parent.onError(t);
    }
    @Override
    public void onComplete() {
    parent.onComplete();
    }
    };

    this.other.subscribe(until);

    return parent;
    }
    }

    We receive the Subscription from the other source (1) and wrap it into a Disposable (similar to how RxJava's Subscription.create() does this now). Due to the terminal nature of Disposables, even if the main chain completes before the other chain has even the chance to receive a Subscription, the SerialDisposable will become disposed and will immediately dispose the other's Subscription.

    Note that since the ability to cancel and thus dispose resources depends on timing and ordering, usually one will end up needing some disposable-container up-front (i.e., sdParent, sdOther) so regardless of which Subscription arrives when, they can cancel out each other as necessary.


    TakeUntil v2

    Looking at our takeUntil() implementation, one can discover that reorganizing the various Subscriptions, it is possible to clear up the mess the Disposables have caused.

        @Override
    // ...
    public Subscriber<? super T> call(Subscriber<? super T> child) {
    Subscriber<T> serial = new SerializedSubscriber<>(child);

    SubscriptionArbiter sa = new SubscriptionArbiter(); // (1)

    DisposableSubscription dsub =
    new DisposableSubscription(sa, new DisposableList()); // (2)

    serial.onSubscribe(dsub); // (3)

    Subscriber<T> parent = new Subscriber<T>() {
    @Override
    public void onSubscribe(Subscription s) {
    dsub.add(Disposables.create(s::cancel)); // (4)
    sa.setSubscription(s); // (5)
    }
    // ...
    };

    Subscriber<Object> until =
    new Subscriber<Object>() {
    @Override
    public void onSubscribe(Subscription s) {
    dsub.add(Disposables.create(s::cancel)); // (6)
    s.request(Long.MAX_VALUE);
    }
    // ...

    It works as follows:

    1. We create the SubscriptionArbiter as before,
    2. then it is wrapped by a DisposableSubscription,
    3. which is then pushed downstream. The pair will make sure cancellation and requests are accumulated until the arbiter receives a 'real'Subscription.
    4. Once the main receives its Subscription, we turn it into a Disposable and add it to the dsub composite,
    5. followed by updating the arbiter's current subscription: any accumulated requests and any cancellation is now 'replayed' to the upstream Subscription.
    6. Once the other source sends its Subscription, we turn it into a Disposable as well and request everything.
    Naturally, the parent will now call dsub.dispose() in its onError() and onComplete() methods.

    Let's think about the various cancellation paths:

    • Downstream's cancellation: it will cancel the dsub, dsub will cancel the arbiter, the arbiter will immediately cancel any Subscription it receives.
    • Main completion: it will cancel the dsub, dsub will cancel the arbiter and the arbiter will cancel the upstream's Subscription. In addition, dsub will immediately cancel the other's subscription if added.
    • Other completion: it will cancel the dsub, dsub will cancel the arbiter. Once the main receives a Subscription, either dsub or the arbiter will cancel it immediately.

    Conclusion

    In this blog post, I've talked about how resource management can be performed with respect to Reactive-Stream's Subscribers and Subscriptions and shown an example implementation of a takeUntil() operator.

    Although it looks like we have to, at least, allocate as many objects similar to RxJava, note that many operators don't really require resource management themselves (example: take()) or don't even need to wrap the Subscription instance in any way (example: map()).

    In the next and final post about the RS API, I'm going to talk about the increased need for various arbiter classes (briefly mentioned in the takeUntil() example), because one can't leave a Subscriber without a Subscription and expect cancellation to work and one can't call onSubscribe() multiple times either if the source changes.

    The Reactive-Streams API (part 4 - final)

    $
    0
    0

    Introduction


    In this final post about the Reactive-Streams API, I'm going to talk about the perhaps surprising trend that one needs a SubscriberArbiter (the cousin of ProducerArbiter) very frequently, especially if there are multiple sources, scheduling or other asynchrony involved.

    In RxJava 1.x, having a Producer set on a Subscriber is optional and one can call onError() and onCompleted() without any Producer. Such calls will eventually call unsubscribe() on the rx.Subscriber itself and clean up any resources.

    In contrast, RS requires a Subscription to be sent through onSubscribe() before one can call the onXXX methods for any reason.

    In this blog post, I'll look at cases where this requirement can cause trouble when the operator is implemented in the classical RxJava structure.

    Deferred subscribing

    An operator which has to consider an error-before-Subscription case is defer(). When subscribing to a deferred Publisher, the operator calls a user factory that returns another Publisher which is then subscribed to. Now since we don't really trust user functions, we need to catch a Throwable exception and notify the child of this problem. However, for this case, we already need a Subscription to be sent to the child Subscriber, but if we already sent one, the generated Producer can't send another one. The solution is to use a SubscriptionArbiter which will allow us to send an early error or switch over to the 'real' source.

    public final class OnSubscribeDefer<T>
    implements OnSubscribe<T> {
    final Func0<? extends Publisher<? extends T>> factory;
    public OnSubscribeDefer(
    Func0<? extends Publisher<? extends T>> factory) {
    this.factory = factory;
    }
    @Override
    public void call(Subscriber<? super T> child) {

    SubscriptionArbiter sa = new SubscriptionArbiter();
    child.onSubscribe(sa); // (1)

    Publisher<? extends T> p;
    try {
    p = factory.call();
    } catch (Throwable e) {
    Exceptions.throwIfFatal(e);
    child.onError(e); // (2)
    return;
    }
    p.subscribe(new Subscriber<T>() { // (3)
    @Override
    public void onSubscribe(Subscription s) {
    sa.setSubscription(s); // (4)
    }

    @Override
    public void onNext(T t) {
    child.onNext(t);
    }

    @Override
    public void onError(Throwable t) {
    child.onError(t);
    }

    @Override
    public void onComplete() {
    child.onComplete();
    }

    });
    }
    }

    This time, we don't need to manage resources, but the Subscription change has to be handled:

    1. We first create an empty arbiter and set it on the child.
    2. In case the function call throws, we can now safely emit the exception through onError because child has received a Subscription in the form of the arbiter.
    3. We can't just use child directly but have to override the onSubscribe method
    4. and set the 'real'Subscription on the arbiter instead. The rest of the methods just delegate directly.

     Time-delayed subscribing

    Let's look at the operator delaySubscription() which delays the actual subscription to the source by some time. We can almost copy-past the current implementation, but it won't compile due to the API changes:


    public final class OnSubscribeDelayTimed<T>
    implements OnSubscribe<T> {
    final Publisher<T> source;
    final Scheduler scheduler;
    final long delay;
    final TimeUnit unit;
    public OnSubscribeDelayTimed(
    Publisher<T> source,
    long delay, TimeUnit unit,
    Scheduler scheduler) {
    this.source = source;
    this.delay = delay;
    this.unit = unit;
    this.scheduler = scheduler;
    }
    @Override
    public void call(Subscriber child) {
    Scheduler.Worker w = scheduler.createWorker();

    // child.add(w);

    w.schedule(() -> {
    // if (!child.isUnsubscribed()) {

    source.subscribe(child);

    // }
    }, delay, unit);
    }
    }


    We can't add a resource to child and we can't check for unsubscription either this way. In order to clear the worker, one needs a disposable-container wrapper, for the cancellation, we need something that can 'replay' cancellation to the Subscription the source is going to emit:


        @Override
    public void call(Subscriber<? super T> child) {
    Scheduler.Worker w = scheduler.createWorker();

    SubscriptionArbiter sa = new SubscriptionArbiter(); // (1)

    DisposableSubscription dsub =
    new DisposableSubscription(
    sa, new DisposableList()); // (2)

    dsub.add(w); // (3)

    child.onSubscribe(dsub); // (4)

    w.schedule(() -> {
    source.subscribe(new Subscriber<T>() { // (5)
    @Override
    public void onSubscribe(Subscription s) {
    sa.setSubscription(s); // (6)
    }

    @Override
    public void onNext(T t) {
    child.onNext(t);
    }

    @Override
    public void onError(Throwable t) {
    child.onError(t);
    }

    @Override
    public void onComplete() {
    child.onComplete();
    }

    });
    }, delay, unit);

    Quite an expanded logic in respect of the original code, but for a good reason:

    1. We need a SubscriptionArbiter because we need cancellation support while the delayed subscription happens, however, we won't have any real Subscription from source yet so we need to hold onto any requests that may come in in the meantime.
    2. We need a way to cancel the schedule in case the child has decided to cancel() the whole operation. Since the arbiter doesn't have resource management, we need to wrap it into a disposable-container. Naturally, we don't really need a list for a single resource and you can implement your own, single resource disposable subscription.
    3. The container will take care of cancellation if we add the Worker instance.
    4. Once the arbiter+disposable are set up, we send the dsub to the child. This way, it can request() and cancel() at any time and both arbiter and dsub will forward the calls to the appropriate (re)sources.
    5. Once the scheduled action runs, we can't just hand the original child Subscriber to it because we already have set a Subscriber on it. Therefore, we need to wrap it with another Subscriber instance that mostly forwards the onXXX calls except onSubscribe.
    6. The onSubscribe needs to change and instead of setting the source Subscription on the child, we set it on the arbiter. The arbiter will then now relay any accumulated requests or cancel the source immediately.
    Now you might think there is an error with the Subscriber (5) because in its onNext methods, it doesn't call sa.produced(1). This causes an accounting inconsistency indeed, but once a real Subscription is set on the arbiter, any future request(n) from child is relayed as-is to the upstream and no further call to setSubscription() will take place in this operator. The upstream receives the correct request amounts and the arbiter just keeps adding them up without having any effect on anything else. To be on the safe side, you can
    • still call sa.produced(1) or
    • implement an arbiter variant which allows only a single Subscription to be set and stops accumulating requests after it is set.

    Conclusion

    In this blog post, I've shown two cases where Subscription arbitration and resource cleanup has to be managed. Luckily, not all operators have to do these behaviors but the majority will, therefore, two specialized Subscriptions are required to handle any delay in a 'real'Subscription and/or the release of resources on cancel().

    Since this is going to happen frequently, I'm sure RxJava 2.0 will provide standard and efficient implementations to help conform to the Reactive-Streams specification.

    As for the RS API series conclusion, the minimal set of interfaces is quite capable of capturing the requirements of a typical dataflow. It is possible to convert RxJava idioms into RS idioms, but the infrastructure and conveniences have to be re-implemented from scratch. I've shown some basic Subscriptions such as SingeSubscription, SingleDelayedSubscription, SubscriptionArbiter and DisposableSubscription that, among other similar new classes will be the main tools to implement operators for RxJava 2.0.

    In the next series, I'm going to cover the most controversial set of objects in the field of reactive programming and I'll not just talk about them in detail but show how to implement your own custom variant of the kind.

    Subjects (part 1)

    $
    0
    0

    Introduction


    I have the feeling many would like to bury Subjects and I'm yet going to do a multi-post series about them.

    Some consider them the mutable state of the reactive world, which I don't disagree with, but then they continue and say in big letters: don't use them while showing some more examples of using Observable.create():

    Observable.create(s -> {
    int i = 0;
    while (true) {
    s.onNext(i++);
    }
    }).subscribe(System.out::println);

    What about unsubscription and backpressure? No amount of operators can fix that for this source, yet you can apply onBackpressureXXX strategies on a Subject at any time.

    Subjects are by no means bad or broken, but as with other components of the reactive paradigm, one must learn when and how to use them. For those who are lecturing about them, they should reconsider when and how they introduce Subjects to their audience. I suggest introducing them after introducing regular fluent operators but before talking about create().

    In this series, I'm going to try and introduce Subjects, detail their requirements and structure and show how one can build its own custom Subject.

    Imperative eventing

    Let's assume you'd like to emit your own events, but you can't be sure when those events will be fired or how many of them will be there. Clearly, just() and from() can't help here and you don't want to create() an Observable which spins on some state either.

    The best would be an object that is both Observable, so clients can chain onto it, and Observer so you can emit values and terminal events as well. The combination of these two is what's called Subject (or Processor in Reactive-Streams naming).

    You can think of them as a way of multicasting your values to interested parties without the hassle to handle thread-safety for the incoming Subscribers.

    Subject changeEvents = ...

    changeEvents.subscribe(System.out::println);
    changeEvents.subscribe(System.out::println);

    changeEvents.onNext("Added");
    Thread.sleep(10);
    changeEvents.onNext("Removed");
    Thread.sleep(10);
    changeEvents.onCompleted();

    Subjects are generified and sometimes you want them to emit the same type as received or emit a completely different type. In C# you can define two kinds of it with different arities:

    interface ISubject<T>: IObserver<T>, IObservable<T> { }

    interface ISubject<T, U>: IObserver<T> IObservable<U> { }

    however, Java's type erasure doesn't let you do this, therefore, RxJava went with the two-parameter version of it:

    abstract class Subject<T, R>
    extends Observable<R> implements Observer<T> { }

    Since RxJava's fluent API entry point is a class itself, in order to retain the composition possibility, Subject has to extend Observable and implement Observer. Perhaps a more interesting option would be to extend Subscriber instead of Observer to gain some resource-management and backpressure options, but Java doesn't let you extend more than one base class either and having the composable methods available is more important.

    So in the example above, you'd have:

    Subject<String, String> changeEvents = ...

    Subjects are also considered hot observables: they 'operate' even if you don't have a Subscriber listening into them, similar to how a radio station doesn't stop playing music if you turn off your radio; who is there is there, who isn't there isn't.

    For the example, such a Subject in RxJava is called a PublishSubject:

    Subject<String, String> changeEvents = PublishSubject.create();

    Certainly, having a create() factory method helps to avoid the repetition of the type parameters, but why can't one just call new PublishSubject<>() like in Rx.NET?

    The reason is how the fluent API is made possible by an Observable class. If you recall, you have to provide an OnSubscribe instance when you create an Observable via its create() method which internally will remember that instance to call call() on it whenever a Subscriber subscribes through subscribe().

    Unlike cold Observables, Subjects have to track their clients (so everyone gets the same data) which tracking has to happen both inside the OnSubscribe and in the Subject body itself. Unfortunately, Java doesn't allow an inner class inside the constructor to access the parent class' fields, therefore, the shared state between the two has to be extracted into a separate class and presented to both of them. This is non-trivial therefore the whole Subject instantiation and setup process is hidden behind such factory methods like PublishSubject.create(). (Once we get to the Subject-construction, I'll explain this with the help of examples.)

    Flavors

    Sometimes, you don't just want to dispatch your events as you see fit and don't care about your subscriber audience.

    For example, you are a TV network and just throwing out a large amount of TV series and their episodes on a weekly basis. However, your clients can't keep up with the sheer amount but they don't want to skip any of the episodes either. Therefore, their smart TV or the cable provider itself offers the capability to cache these episodes, starting at some point, and allow the subscriber to watch it in sequence in its own pace and without leaving out any.

    In the programming world, you'd want to emit events and allow clients to run at a different pace but still receive all of your events, perhaps even after you stopped generating those events: a late Subscriber comes in and you'd like to replay all events that has been accumulated during the active times.

    This is called a ReplaySubject.

    By default, you can create an unbounded ReplaySubject which will cache everything it receives and replays it to Subscribers, including any terminal events.

    However, some use cases require limiting the retention time or amount of the cached elements so later Subscribers don't get everything from the very beginning. The RxJava API offers three additional replay modes:

    • createWithSize(n) will retain at most n elements,
    • createWithTime(t, u) will retain elements that are younger that than t and
    • createWithTimeAndSize(n, t, u) will retain at most n elements which are also younger than t.
    This could be enough, but there exist further cases that warrant for an unique Subject implementation.

    For example, if one performs an asynchronous computation and wants to emit just a single value followed by a completion event. ReplaySubject does work but is too verbose and the overhead might be inacceptable for such a simple task. Therefore, RxJava offers another Subject called AsyncSubject. It remembers the very last element it received and once onCompleted is called, all currently listening and future listeners will receive just that single value followed by the completion event. But unlike ReplaySubject, if one calls onError on an AsyncSubject, any previously received value is ignored and all Subscribers will just receive the Throwable from then on.

    The final case that is covered in RxJava via a Subject variant is when one would like to have a single value stored. Subscribers to it should immediately receive this value and any subsequent value in case a new value is onNext'd on the Subject. It is called BehaviorSubject and some also call it the reactive property. Again, a ReplaySubject of size 1 would also do, but unlike ReplaySubject, sending an onCompleted() will evict the single stored value and subsequent Subscribers will only receive just the onCompleted (or onError) event and not any value. You can create a BehaviorSubject with or without an initial value.

    A reactive list

    After such dry text, let's see how we can put these subjects into use. Imagine you want to build a list class that sends out notifications whenever its contents change. We want to send out events in case of addition, removal or update to some element and have

    • a notification channel for the type of change,
    • a notification channel of the value causing the change, which also remembers the last value that was involved in a change and
    • a notification channel that holds onto the last 10 elements added to the list.
    I'll call this class ReactiveList:

    public final class ReactiveList<T> {

    public enum ChangeType {
    ADD, REMOVE, UPDATE // (1)
    };

    final List<T> list = new ArrayList<>(); // (2)

    final PublishSubject<Changetype> changes =
    PublishSubject.create(); // (3)
    final BehaviorSubject<T> changeValues =
    BehaviorSubject.create();
    final ReplaySubject<T> latestAdded =
    ReplaySubject.createWithSize(10);

    public Observable<ChangeType> changes() { // (4)
    return changes;
    }

    public Observable<T> changeValues() {
    return changeValues;
    }

    public Observable<T> latestAdded() {
    return latestAdded;
    }

    public void add(T value) {
    // implement
    }
    public void remove(T value) {
    // implement
    }
    public void replace(T value, T newValue) {
    // implement
    }
    }

    Our ReactiveList consist of the following notable elements:
    1. We define an enum for the three kinds of change events: add, remove and update.
    2. The data will be stored in a regular j.u.List instance.
    3. We specify the Subjects for the various event outputs and
    4. we specify methods that return an Observable of those outputs.
    Now let's see the implementation of the mutation methods:


    // ...
    public void add(T value) {
    list.add(value);
    changes.onNext(ChangeType.ADD);
    changeValues.onNext(value);
    latestAdded.onNext(value);
    }
    public void remove(T value) {
    if (list.remove(value)) {
    changes.onNext(ChangeType.REMOVE);
    changeValues.onNext(value);
    }
    }
    public void replace(T value, T newValue) {
    int index = list.indexOf(value);
    if (index >= 0) {
    list.set(index, newValue);
    changes.onNext(ChangeType.UPDATE);
    changeValues.onNext(newValue);
    }
    }
    }


    Simply put, we perform the relevant changes and call onNext on the subjects to propagate the relevant change information.

    A more reactive list

    What if we want the inputs to the ReactiveList to be reactive as well by providing an Observer surface for them instead?

    For simplicity, I'll drop the replace() functionality but add a new method list() that will return a snapshot Observable of the contents of the list at the call time.

    Now that we go for full reactivity, one has to consider concurrent calls to the Subjects. Subjects implement the Observer interface and thus inherit the requirement that onXXX methods have to be called in a sequential fashion and not concurrently. We can, of course use what we learned from serialized access, but such behavior is so often required the Subject has a dedicated operator for it, toSerialized() which returns a Subject that can be accessed fully concurrently.

    public class MoreReactiveList<T> {
    public enum ChangeType {
    ADD, REMOVE
    };

    final List<T> list = new ArrayList<>();

    final Subject<ChangeType, ChangeType> changes; // (1)
    final Subject<T, T> changeValues;
    final Observer<T> addObserver; // (2)
    final Observer<T> removeObserver;

    public Observable<ChangeType> changes() {
    return changes;
    }

    public Observable<T> changeValues() {
    return changeValues;
    }

    public Observable<T> list() { // (3)
    List<T> copy = new ArrayList<>();
    synchronized (list) {
    copy.addAll(list);
    }
    return Observable.from(copy);
    }

    public Observer<T> adder() { // (4)
    return addObserver;
    }

    public Observer<T> remover() {
    return removeObserver;
    }

    void onAdd(T value) { // (5)
    synchronized (list) {
    list.add(value);
    }
    changes.onNext(ChangeType.ADD);
    changeValues.onNext(value);
    }

    void onRemove(T value) {
    synchronized (list) {
    if (!list.remove(value)) {
    return;
    }
    }
    changes.onNext(ChangeType.REMOVE);
    changeValues.onNext(value);
    }

    void clear() {
    synchronized (list) {
    list.clear();
    }
    }

    public MoreReactiveList() {
    // implement
    }
    }

    The data types and structure has changed a little:

    1. Since we are going to serialize the Subjects, we lose their concrete type and have to specify them as Subject<T, R>.
    2. We will have a single set Observers for both add- and remove-channels.
    3. Here we have the new a list() method that creates a snapshot of the underlying list and returns an Observable of it. Note the synchronization around the list itself: the class will be manipulated concurrently and we need to ensure thread safety.
    4. We return the final Observer instances for the add- and remove-channels through the methods.
    5. The actual list manipulation has been moved to the package-private onAdd and onRemove methods.
    Now let's see the missing MoreReactiveList() constructor's logic:

        // ...
    public MoreReactiveList() {
    changes =
    PublishSubject.<ChangeType>create()
    .toSerialized(); // (1)

    changeValues =
    BehaviorSubject.<T>create()
    .toSerialized();

    addObserver = new SerializedObserver<>( // (2)
    Observers.create(
    this::onAdd,
    t -> {
    clear();
    changes.onError(t);
    changeValues.onError(t);
    },
    () -> {
    clear();
    changes.onCompleted();
    changeValues.onCompleted();
    }
    ));
    removeObserver = new SerializedObserver<>( // (3)
    Observers.create(
    this::onRemove,
    t -> {
    clear();
    changes.onError(t);
    changeValues.onError(t);
    },
    () -> {
    clear();
    changes.onCompleted();
    changeValues.onCompleted();
    }
    ));
    }
    }

    It works as follows:

    1. We setup the output channels similar to ReactiveList but this time, we serialize the access to them.
    2. The addObserver is a serialized observer that forwards its onNext method to the onAdd method. If it receives a terminal event, the list is cleared and the event is propagated to both output channels.
    3. The removeObserver is built up similarly to (2) with the exception that it forwards to onRemove.
    Let's try it out:

    MoreReactiveList<Long> list = new MoreReactiveList<>();

    Observable.timer(0, 1, TimeUnit.SECONDS)
    .take(10)
    .subscribe(list.adder());

    Observable.timer(4, 1, TimeUnit.SECONDS)
    .take(10)
    .subscribe(list.remover());

    list.changes().subscribe(System.out::println);

    list.changes()
    .flatMap(e -> list.list().toList())
    .subscribe(System.out::println);

    list.changeValues.toBlocking().forEach(System.out::println);

    In the example, we create a list and two timers which run 4 seconds apart and emit items every second. We use the first timer to add to the list and the second to remove from the list. We then observe the change types, we print the current state of the list at when the contents change and we blockingly observe the change-causing values themselves until completion.

    Conclusion

    In this blog post, I've introduced the Subjects and their RxJava flavors, then shown two reactive objects that make use of their capabilities.

    In the next post, I'll explain the requirements, structures and algorithms that are necessary to implement a custom Subject.

    Subjects (part 2)

    $
    0
    0

    Introduction


    Sorry for the delay in posting, I was busy with my professional work and I've been implementing a fully reactive-streams compliant RxJava 2.0 in the meantime.

    In this blog post, I'm going to talk about the requirements of building Subjects, the related structures and algorithms and I'm going to build a backpressure-aware special subject, UnicastSubject, with them.

    Requirements

    Since subjects implement both Observer and Observable, they have to conform to both:

    • [Observer] onXXX events have to be sequential and expected to be sequential
    • [Observer] should conform the pattern onNext* (onError | onCompleted)?
    • [Observable] subscribing to it should be thread-safe
    In addition, since subjects can reach a terminal state via onError() or onCompleted(), we must deal with the situation when a Subscriber subscribes to a Subject after such event. Clearly, keeping the Subscriber hanging at this point isn't a good idea. The standard RxJava subjects, therefore, re-emit their terminal event to such late-commers (ReplaySubject may emit onNext events before that though).

    Given that, we want the UnicastSubject to allow only a single Subscriber, buffer incoming events until this single Subscriber subscribes and replay/relay events while conforming to the backpressure requests of the Subscriber.

    Luckily, we already saw all components needed for implementing the UnicastSubject: tracking the Subscriber's presence and using queue-drain to replay/relay events to it.



    UnicastSubject


    Let's start by building a skeleton class UnicastSubject:

    public final class UnicastSubject<T> 
    extends Subject<T, T> {

    public static <T> UnicastSubject<T> create() { // (1)
    State<T> state = new State<>();

    return new UnicastSubject<>(state);
    }

    final State<T> state; // (2)

    protected UnicastSubject(State<T> state) { // (3)
    super(state);
    this.state = state;
    }

    @Override
    public void onNext(T t) {
    // implement
    }

    @Override
    public void onError(Throwable e) {
    // implement
    }

    @Override
    public void onCompleted() {
    // implement
    }

    @Override
    public boolean hasObservers() {
    // implement
    }

    static class State<T> implements
    OnSubscribe<T>, Observer<T>,
    Producer, Subscription { // (4)

    }
    }

    The main drawback of the Java language regarding RxJava is that there are no extension methods: methods that appear to be part of a class but in reality, they are static methods somewhere else and the compiler turns a fluent invocation to them into a regular imperative static method call. Therefore, a fluent API requires a wrapper class, Observable, to hold onto all operators and methods.

    Since we need to customize the subscription actions for each custom Observable, the Observable class has a protected constructor with a callback, OnSubscribe<T> on it. However, Subjects need to both handle the OnSubscribe<T> calls and the onXXX methods at the same time. Java forbids calling instance methods before the constructor calls super, therefore, the following code doesn't compile:

    public class MyObservable<T> extends Observable<T> {
    protected MyObservable() {
    super(this::handleSubscriber);
    }

    void handleSubscriber(Subscriber<? super T> s) {
    }
    }

    The workaround is somewhat awkward, but nonetheless working: use static factory methods to create Subjects and have a shared state object between which is then used as an OnSubscribe<T> target and serves as the state of the Observable itself (1).

    Note that most 1.x Observable extensions, such as the standard Subjects use separate objects for state and subscription handling. 2.x has been an improvement in this regard and Subjects also use a single state object for both tasks, similar to what I'm showing here.

    Given a single State<T> object, we use it as the OnSubscribe<T> callback and store it inside the UnicastSubject (2, 3). The State class itself implements a bunch of interfaces (4):


    • OnSubscribe for handling subscription, 
    • Observer for conveniently have onXXX methods that will be delegated to from the UnicastSubject.onXXX methods, 
    • Producer, since we know there will be only a single Subscriber which needs only a single Producer to communicate with (saving on allocation and on cross-communication) and
    • Subscription for handling the unsubscription call coming from the child directly (again saving on allocation and cross-communication).
    The implementation of the main onXXX methods is straightforward delegation into the state object:

        // ...
    @Override
    public void onNext(T t) {
    state.onNext(t);
    }

    @Override
    public void onError(Throwable e) {
    state.onError(e);
    }

    @Override
    public void onCompleted() {
    state.onCompleted();
    }

    @Override
    public boolean hasObservers() {
    return state.child != null;
    }
    // ...


    (For brevity, I'm not going to implement the Subject state peeking methods in this post.)

    Now let's build the State class. First, I'll add the state variables:


    static final class State<T> implements 
    OnSubscribe<T>, Observer<T>, Producer, Subscription {

    volatile Subscriber<? super T> child; // (1)

    final AtomicBoolean once = new AtomicBoolean(); // (2)

    final Queue<T> queue = new SpscLinkedAtomicQueue<>(); // (3)

    volatile boolean done; // (4)
    Throwable error;

    volatile boolean unsubscribed; // (5)

    final AtomicLong requested = new AtomicLong(); // (6)

    final AtomicInteger wip = new AtomicInteger(); // (7)
    // ...

    We have a couple of components here:

    1. The child field will hold onto the only subscriber when it arrives and will be set back to null once it leaves. It has to be volatile because hasObservers() needs to check it in thread-safe manner.
    2. We need to make sure only a single Subscriber is let in during the entire lifetime of the subject and we do this via an atomic boolean. Naturally, this field could have been inlined into State by StateextendsAtomicBoolean. An alternative, although requires more work would have been to use child directly and have a private static Subscriber instance that indicates a terminal state.
    3. This queue will hold onto the incoming values until there is a Subscriber or said subscriber requests some elements. Here, I'm using a single-producer single-consumer linked queue implementation from RxJava, but this queue is relatively expensive due to constant node allocation. In an upcoming PR and in RxJava 2.x, we have a SpscLinkedArrayQueue (it is a slightly modified version of the JCTools SpscUnboundedArrayQueue) that amortizes this allocation by using 'islands' of Spsc buffers.
    4. We hold onto the terminal event (which might be an error) in these two fields. Since error will be written once before done and read after done, it doesn't need to be a volatile in itself.
    5. Since the child may go at any time, it would be infeasible to keep buffering events indefinitely since no one else could observe them after that. This flag, along with the done flag will be used in the onXXX methods to drop events.
    6. We need to keep track of the requested amount of the child subscriber so we emit values no more than requested.
    7. The wip field is part of the queue-drain approach explained in an earlier post and makes sure only a single thread is emitting values to the child subscriber, if any.
    In the next step, let's implement the call() method of OnSubscribe and manage the incoming Subscribers:


    @Override
    public void call(Subscriber<? super T> t) {
    if (!once.get() && once.compareAndSet(false, true)) { // (1)
    t.add(this);
    t.setProducer(this); // (2)
    child = t;
    drain(); // (3)
    } else {
    if (done) { // (4)
    Throwable e = error;
    if (e != null) { // (5)
    t.onError(e);
    } else {
    t.onCompleted();
    }
    } else {
    t.onError(new IllegalStateException(
    "Only one subscriber allowed.")); // (6)
    }
    }
    }


    1. If once is not set and we succeed setting it to true atomically, we now have our single child subscriber.
    2. It is important that setting up the unsubscription callback and the producer happens before the store to the child field, otherwise, an asynchronous onNext call may end up running before this setup. This isn't much of a problem in RxJava 1.x but it is of a large concern in a reactive-streams compliant Publisher (to which we'd like to port our code more easily.)
    3. Once the producer/unsubscription are set up, we set the child reference. Whether or not the child is cancelled at this point, we need to call drain which will take care of replaying buffered values and cleaning up if the child has unsubscribed in the meantime.
    4. If there is/was a subscriber already there, we need to check if the subject is actually terminated.
    5. If the subject is terminated, we simply emit the terminal event (error or completion), similar to how the standard subjects behave.
    6. Otherwise, if there is a subscriber and the subject isn't terminated, we simple emit an IllegalStateException explaining the situation.
    Next comes onNext, which could be implemented in a simple way and in a more complicated way. Let's see the simpler way:


    @Override
    public void onNext(T t) {
    if (done || unsubscribed) {
    return;
    }
    queue.offer(t);
    drain();
    }

    Not very exciting, is it? If the subject isn't done or isn't unsubscribed, offer the value to the queue and call drain(). (Note that I'm omitting the null handling here for brevity, again).

    The complicated way is to have a fast-path implementation that bypasses the queue if there is no contention, but more importantly, when the child has caught up and thus the queue is empty. Note, however, that such caught-up state is not permanent because the child can slow down and not request in time. In this case, we still have to put the value into the queue.


    @Override
    public void onNext(T t) {
    if (done || unsubscribed) {
    return;
    }
    if (wip.get() == 0 && wip.compareAndSet(0, 1)) { // (1)
    long r = requested.get(); // (2)
    if (r != 0 && queue.isEmpty()) { // (3)
    child.onNext(t);
    if (r != Long.MAX_VALUE) { // (4)
    requested.decrementAndGet();
    }
    if (wip.decrementAndGet() == 0) { // (5)
    return;
    }
    } else {
    queue.offer(t); // (6)
    }
    } else {
    queue.offer(t); // (7)
    if (wip.getAndIncrement() != 0) {
    return;
    }
    }
    drainLoop(); // (8)
    }

    This looks more interesting. Here is how it works:

    1. This is the entry to the fast-path; if we manage to set the wip counter from 0 to 1, we are in.
    2. We retrieve the requested amount.
    3. We need to check if the requested amount is non-zero and if the queue is empty. This queue check is crucial because otherwise the fast-path would skip the contents of the queue and thus reorder the events.
    4. If the queue happens to be empty and the child runs in bounded mode, we emit the event and decrement the requested amount.
    5. We also decrement the wip count and if it is zero, we simply return. If it is non-zero, it means a concurrent request() call arrived and we need to deal with it. The execution stay's in "emission mode" and continues on line (8)
    6. If there is no request or the queue is non-empty, we enqueue the current value and let the drainLoop() deal with the situation (8).
    7. If we couldn't enter the fast-path, we offer the value into the queue and try to enter the drain loop by incrementing wip. If it was actually zero, we enter the drain loop, otherwise, we did indicate a running drain loop there is more work to do.
    8. Finally, while still in "emission mode", we jump to the drain loop.

    The implementation of the onError and onCompleted isn't that different from the simpler onNext implementation:

    @Override
    public void onError(Throwable e) {
    if (done || unsubscribed) {
    return;
    }
    error = e;
    done = true;
    drain();
    }

    @Override
    public void onCompleted() {
    if (done || unsubscribed) {
    return;
    }
    done = true;
    drain();
    }

    They are quite straightforward: unless done or unsubscribed, set an error, then done and then call drain().

    Handling child requests and unsubscription isn't that complicated either:


    @Override
    public void request(long n) {
    if (n < 0) {
    throw new IllegalArgumentException("n >= 0 required");
    }
    if (n > 0) {
    BackpressureUtils.getAndAddRequest(requested, n); // (1)
    drain();
    }
    }

    @Override
    public boolean isUnsubscribed() {
    return unsubscribed;
    }

    @Override
    public void unsubscribe() {
    if (!unsubscribed) {
    unsubscribed = true;
    if (wip.getAndIncrement() == 0) { // (2)
    clear();
    }
    }
    }

    In request() (1), we add the amount to the requested value and then call drain. The unsubscribe() is a bit more interesting (2). We set the unsubscribed flag (the get and then set isn't atomic, but it doesn't matter here) then increment the wip counter, which may seem odd. The idea here is that if there is no contention, the transition from 0 to 1 makes sure the cleanup code runs only once and also prevents any further attempts to enter the emission loop by other means. If there is a drain loop running, this will indicate more work is available and since drainLoop() will check for unsubscription before any other action, drainLoop() will call the cleanup for us.

    Moving on, the clear() method and the drain() methods are also short:


    void clear() {
    queue.clear();
    child = null;
    }

    void drain() {
    if (wip.getAndIncrement() == 0) {
    drainLoop();
    }
    }

    The clear() method clears the queue and nulls out the child. Since this only runs while wip != 0, there is no chance the child gets null and onNext tries to invoke a method on it, yielding NullPointerException. The drain() method simply increments wip and if it was zero, enters the drainLoop().

    Okay, this is what we've been waiting for, the drainLoop method:


    void drainLoop() {
    int missed = 1; // (1)

    final Queue<T> q = queue;
    Subscriber<? super T> child = this.child; // (2)

    for (;;) {

    if (child != null) { // (3)

    if (checkTerminated(done, q.isEmpty(), child)) { // (4)
    return;
    }

    long r = requested.get();
    boolean unbounded = r == Long.MAX_VALUE;
    long e = 0L; // (5)

    while (r != 0L) {
    boolean d = done;
    T v = q.poll();
    boolean empty = v == null; // (6)

    if (checkTerminated(d, empty, child)) {
    return;
    }

    if (empty) {
    break;
    }

    child.onNext(v);

    r--;
    e--; // (7)
    }

    if (e != 0) {
    if (!unbounded) {
    requested.addAndGet(e); // (8)
    }
    }
    }

    missed = wip.addAndGet(-missed); // (9)
    if (missed == 0) {
    return;
    }

    if (child == null) { // (10)
    child = this.child;
    }
    }
    }

    Should look quite familiar. Let's dig into it:

    1. Instead of decrementing the wip counter one by one, we fist assume we only missed 1 drain call. Later on, if we happened to miss multiple calls, missed will be bigger and we'll subtract all of them at once on line (9) reducing the likelihood of looping too many times.
    2. We cache the child and queue fields to avoid re-reading them all the time.
    3. If we don't have a child subscriber, there is nothing we can do as of then.
    4. We have to check if a terminal state has been reached. Since onError and onCompleted travel without requests, we need to do the check before checking the requested amount and quit the loop if so. It is important to remember that checking the done flag before the emptiness of the queue is mandatory, because values added to the queue happen before the setting of the done flag.
    5. We read the requested amount, check if it is unbounded and prepare an emission counter.
    6. We read the done flag, poll an element from the queue and if that value is null, we set an empty flag. The call to checkTerminated, again makes sure the unsubscription and terminal events are handled.
    7. We decrement the requested amount and the emission count. By decrementing instead of incrementing, we save a negation on line (8).
    8. If there was emission and the request amount is not unbounded, we decrement the requested amount be the emission amount (e is negative).
    9. Once all that could be done emission-wise, we update the wip counter by subtracting the known missed amount. It is possible more calls were missed and thus the return value won't be zero. In this case, we loop again, otherwise we quit the drain loop.
    10. If we missed an event, it is possible a subscriber arrived thus the child field has to be re-read if we know it as being null locally.
    The final method is the checkTerminated and we are done. Depending on one wants to delay error or not, there are two implementations possible. If errors should be delayed, the method looks like this:

    boolean checkTerminated(boolean done, boolean empty, 
    Subscriber<? super T> child) {
    if (unsubscribed) { // (1)
    clear();
    return true;
    }
    if (done && empty) { // (2)
    unsubscribed = true;
    this.child = null;
    Throwable e = error;
    if (e != null) {
    child.onError(e);
    } else {
    child.onCompleted();
    }
    return true;
    }
    return false;
    }

    Here, we first check for the unsubscribed flag, if set, we clear the queue, null out the child field and indicate the loop should simply quit (1). Otherwise, we check if the source is done and the queue is empty, at which point we set the unsubscribed flag (for convenience), null out the child and emit the terminal event. Any other case will keep the loop running (i.e., done but queue not empty).

    The alternative implementation sends an error as soon as it is detected, ignoring the contents of the queue.


    boolean checkTerminated(boolean done, boolean empty, 
    Subscriber<? super T> child) {
    if (unsubscribed) {
    clear();
    return true;
    }
    if (done) {
    Throwable e = error; // (1)
    if (e != null) {
    unsubscribed = true;
    clear();
    child.onError(e);
    return true;
    } else
    if (empty) { // (2)
    unsubscribed = true;
    this.child = null;
    child.onCompleted();
    return true;
    }
    }
    return false;
    }

    If the done flag is set, we check if there is an associated error as well. In this case, we clear the queue, null out the child via clear() and emit the error (1). Otherwise, we need to check if the queue is empty since we first have to emit the events and only then complete the child (2). There is no need to clear the queue since we know it to be empty at this point, hence no call to clear().


    Conclusion

    In this blog post, I've shown how to build a custom UnicastSubject that honors backpressure and allows only a single Subscriber to subscribe to it.

    In the next blog post, I'll show how to handle multiple Subscribers within a subject by re-implementing the PublishSubject.

    Subjects (part 3 - final)

    $
    0
    0

    Introduction


    In this final part about Subjects, I'll show how to implement a PublishSubject. To make things a bit interesting, I'll implement it in a way that prevents PublishSubject overflowing its child subscribers in case they happen to be not requesting fast enough.


    PublishSubject

    The main body of the PublishSubject will look very similar to UnicastSubject from the last post, therefore, I'm going to skip those details and focus on the State class.

    The State class will look differently because in PublishSubject, there could be many Subscribers at once, each with its own unsubscribed, requested and wip counters. Therefore State can't implement Producer and Subscription anymore. We need to implement those in a new class, SubscriberState, and have an instance for each child Subscriber.

    One thing before detailing the classes, namely the backpressure handling strategy. We want to give the user the ability to specify the backpressure-behavior of the PublishSubject, so he/she doesn't have to apply onBackpressureXXX methods on the output. For this, I define an enum with 3 values:


    enum BackpressureStrategy {
    DROP,
    BUFFER,
    ERROR
    }

    public static <T> PublishSubject&ltT> createWith(
    BackpressureStrategy strategy) {
    State&ltT> state = new State<>(strategy);
    return new PublishSubject<>(state);
    }

    The values should be self-explanatory: drop values, buffer values until the child can consume it or kick out the child Subscriber with an error.

    Now let's see the skeleton of the State class:


    static final class State<T> 
    implements OnSubscribe<T>, Observer<T> {
    final BackpressureStrategy strategy;

    @SuppressWarnings("unchecked")
    volatile SubscriberState<T>[] subscribers = EMPTY;

    @SuppressWarnings("rawtypes")
    static final SubscriberState[] EMPTY = new SubscriberState[0];

    @SuppressWarnings("rawtypes")
    static final SubscriberState[] TERMINATED =
    new SubscriberState[0];

    volatile boolean done;
    Throwable error;

    public State(BackpressureStrategy strategy) {
    this.strategy = strategy;
    }

    boolean add(SubscriberState<T> subscriber) {
    // TODO Auto-generated method stub

    }

    void remove(SubscriberState<T> subscriber) {
    // TODO Auto-generated method stub

    }

    Subscriber<T>[] terminate() {
    // TODO Auto-generated method stub

    }


    @Override
    public void call(Subscriber<? super T> t) {
    // TODO Auto-generated method stub

    }

    @Override
    public void onNext(T t) {
    // TODO Auto-generated method stub

    }

    @Override
    public void onError(Throwable e) {
    // TODO Auto-generated method stub

    }

    @Override
    public void onCompleted() {
    // TODO Auto-generated method stub

    }
    }

    Nothing standing out so far. We have a volatile array of SubscriberState instances, add, remove and terminate methods. By using EMPTY, we will avoid allocating an empty array whenever all subscribers unsubscribe. This pattern should be familiar from an earlier post about Subscription containers. Now let's see the implementation of add().


    boolean add(SubscriberState<T> subscriber) {
    synchronized (this) {
    SubscriberState<T>[] a = subscribers;
    if (a == TERMINATED) {
    return false;
    }
    int n = a.length;

    @SuppressWarnings("unchecked")
    SubscriberState<T>[] b = new SubscriberState[n + 1];

    System.arraycopy(a, 0, b, 0, n);
    b[n] = subscriber;
    subscribers = b;
    return true;
    }
    }


    For the sake of diversity, the State class will use synchronized instead of an atomic CAS loop. The block is essentially a copy-on-write implementation. The benefit of such implementation is that looping through the current array of subscribers is faster and relies on the observation that many Subjects don't actually serve too many Subscribers at once. If, however, one encounters a case where the number of subscribers is large, one can use any list or set based container inside the block instead. The drawback there is that one needs a safe way to iterate over the collection which may only be possible by doing a defensive copy all the time.

    Let's see the remove() method:

    @SuppressWarnings("unchecked")
    void remove(SubscriberState<T> subscriber) {
    synchronized (this) {
    SubscriberState<T>[] a = subscribers;
    if (a == TERMINATED || a == EMPTY) {
    return;
    }
    int n = a.length;

    int j = -1;
    for (int i = 0; i < n; i++) {
    if (a[i] == subscriber) {
    j = i;
    break;
    }
    }

    if (j < 0) {
    return;
    }
    SubscriberState<T>[] b;
    if (n == 1) {
    b = EMPTY;
    } else {
    b = new SubscriberState[n - 1];
    System.arraycopy(a, 0, b, 0, j);
    System.arraycopy(a, j + 1, b, j, n - j - 1);
    }
    subscribers = b;
    }
    }

    Again, it is a copy-on-write implementation with the reuse of the empty array.

    Next comes the terimate() method:


    @SuppressWarnings("unchecked")
    SubscriberState<T>[] terminate() {
    synchronized (this) {
    SubscriberState<T>[] a = subscribers;
    if (a != TERMINATED) {
    subscribers = TERMINATED;
    }
    return a;
    }
    }

    Here, we check if the current state isn't terminated and if so, we set the terminated array and return the last array of known subscribers.

    Implementing the call() method is now possible:


    @Override
    public void call(Subscriber<? super T> t) {
    SubscriberState<T> innerState =
    new SubscriberState<>(t, this); // (1)
    t.add(innerState); // (2)
    t.setProducer(innerState);

    if (add(innerState)) { // (3)
    if (strategy == BackpressureStrategy.BUFFER) { // (4)
    innerState.drain();
    } else
    if (innerState.unsubscribed) { // (5)
    remove(innerState);
    }
    } else {
    Throwable e = error; // (6)
    if (e != null) {
    t.onError(e);
    } else {
    t.onCompleted();
    }
    }
    }


    1. We create and wrap the child subscriber into a SubscriberState that will manage the dispatching of events for each subscriber individually.
    2. We add it to the child subscriber as a Subscription for unsubscription and request handling.
    3. We try to add the innerState object into the array of subscribers, which may fail if the subject is concurrently terminating.
    4. In case we run in a buffering mode, we need to start draining the buffer.
    5. Even if the add() succeeded, the child might have unsubscribed concurrently and the subscribers array may still contain a reference to it. In this case, we explicitly try to remove it again.
    6. If the add() failed on line (3), it means the subject reached its terminal state and we need to emit the terminal event to the child.
    Implementing the onXXX methods are relatively simple, all conform to a similar pattern:

    @Override
    public void onNext(T t) {
    if (done) {
    return;
    }
    for (SubscriberState<T> innerState : subscribers) {
    innerState.onNext(t);
    }
    }

    @Override
    public void onError(Throwable e) {
    if (done) {
    return;
    }
    error = e;
    done = true;
    for (SubscriberState<T> innerState : terminate()) {
    innerState.onError(e);
    }
    }

    @Override
    public void onCompleted() {
    if (done) {
    return;
    }
    done = true;
    for (SubscriberState<T> innerState : terminate()) {
    innerState.onCompleted();
    }
    }

    They simply loop over the known subscribers and emit the event to them.

    So far, we have just forwarded the event handling to another class. It's time we handle the event delivery to the actual child subscriber in SubscriberState.


    static final class SubscriberState<T> 
    implements Producer, Subscription, Observer<T> {
    final Subscriber<? super T> child; // (1)
    final State<T> state; // (2)
    final BackpressureStrategy strategy; // (3)

    final AtomicLong requested = new AtomicLong(); // (4)

    final AtomicInteger wip = new AtomicInteger(); // (5)

    volatile boolean unsubscribed; // (6)

    volatile boolean done;
    Throwable error;

    final Queue<T> queue; // (7)

    public SubscriberState(
    Subscriber<? super T> child, State<T> state) {
    this.child = child;
    this.state = state;
    this.strategy = state.strategy;
    Queue<T> q = null;
    if (strategy == BackpressureStrategy.BUFFER) { // (8)
    q = new SpscLinkedAtomicQueue<>();
    }
    this.queue = q;
    }

    @Override
    public void onNext(T t) {
    // TODO Auto-generated method stub
    }

    @Override
    public void onError(Throwable e) {
    // TODO Auto-generated method stub
    }

    @Override
    public void onCompleted() {
    // TODO Auto-generated method stub
    }

    @Override
    public void request(long n) {
    // TODO Auto-generated method stub
    }

    @Override
    public boolean isUnsubscribed() {
    return unsubscribed;
    }

    @Override
    public void unsubscribe() {
    // TODO Auto-generated method stub
    }

    void drain() {
    // TODO Auto-generated method stub
    }
    }

    1. We keep a reference to the actual subscriber,
    2. we want to remove the SubscriberState in case the child unsubscribes from the state,
    3. we keep a local reference to the BackpressureStrategy to avoid one dereference,
    4. we track the requested amount,
    5. we want to implement a queue-drain logic and require a work-in-progress indicator, 
    6. we want to know if the child called unsubscribe(),
    7. in case the backpressure strategy is BUFFER, we need to store values temporarily and
    8. finally, we instantiate the queue only if the strategy is BUFFER.
    Now let's see the unimplemented methods one by one:


    @Override
    public void onNext(T t) {
    if (unsubscribed) {
    return;
    }
    switch (strategy) {
    case BUFFER:
    queue.offer(t); // (1)
    drain();
    break;
    case DROP: {
    long r = requested.get(); // (2)
    if (r != 0L) {
    child.onNext(t);
    if (r != Long.MAX_VALUE) {
    requested.decrementAndGet();
    }
    }
    break;
    }
    case ERROR: {
    long r = requested.get(); // (3)
    if (r != 0L) {
    child.onNext(t);
    if (r != Long.MAX_VALUE) {
    requested.decrementAndGet();
    }
    } else {
    unsubscribe();
    child.onError(
    new MissingBackpressureException());
    }

    break;
    }
    default:
    }
    }

    The method looks complicated because it handles all strategies together:

    1. When running in BUFFER mode, we queue the value and call drain.
    2. In DROP mode, we check and decrement the requested amount and drop values if there are no requests.
    3. In ERROR mode, if the requested amount is zero, we unsubscribe and send out a MissingBackpressureException.
    The implementation of onError() and onCompleted(), again, is really similar and nothing complicated:

    @Override
    public void onError(Throwable e) {
    if (unsubscribed) {
    return;
    }
    if (strategy == BackpressureStrategy.BUFFER) {
    error = e;
    done = true;
    drain();
    } else {
    child.onError(e);
    }
    }

    @Override
    public void onCompleted() {
    if (unsubscribed) {
    return;
    }
    if (strategy == BackpressureStrategy.BUFFER) {
    done = true;
    drain();
    } else {
    child.onCompleted();
    }
    }

    The next three methods, request(), isUnsubscribed() and unsubscribed() should be familiar to you:


    @Override
    public void request(long n) {
    if (n < 0) {
    throw new IllegalArgumentException();
    }
    if (n > 0) {
    BackpressureUtils.getAndAddRequest(requested, n);
    if (strategy == BackpressureStrategy.BUFFER) {
    drain();
    }
    }
    }

    @Override
    public boolean isUnsubscribed() {
    return unsubscribed;
    }

    @Override
    public void unsubscribe() {
    if (!unsubscribed) {
    unsubscribed = true;
    state.remove(this);
    if (strategy == BackpressureStrategy.BUFFER) {
    if (wip.getAndIncrement() == 0) {
    queue.clear();
    }
    }
    }
    }

    We only need to call drain and clear the queue in case we run in a BUFFER mode.

    Last but not least, let's see the drain() method.


    void drain() {
    if (wip.getAndIncrement() != 0) {
    return;
    }

    int missed = 1;

    Queue<> q = queue;
    Subscriber child = this.child;

    for (;;) {

    if (checkTerminated(done, q.isEmpty(), child)) {
    return;
    }

    long r = requested.get();
    boolean unbounded = r == Long.MAX_VALUE;
    long e = 0L;

    while (r != 0) {
    boolean d = done;
    T v = q.poll();
    boolean empty = v == null;

    if (checkTerminated(d, empty, child)) {
    return;
    }

    if (empty) {
    break;
    }

    child.onNext(v);

    r--;
    e--;
    }

    if (e != 0) {
    if (!unbounded) {
    requested.addAndGet(e);
    }
    }

    missed = wip.addAndGet(-missed);
    if (missed == 0) {
    return;
    }
    }
    }

    The drain loop should look quite familiar at this point. Nothing new or special here.

    Finally, let's see the checkTerminated() method which now has the responsibility to clean up.


    boolean checkTerminated(boolean done, 
    boolean empty,
    Subscriber<? super T> child) {
    if (unsubscribed) {
    queue.clear(); // (1)
    state.remove(this);
    return true;
    }
    if (done && empty) {
    unsubscribed = true; // (2)
    Throwable e = error;
    if (e != null) {
    child.onError(e);
    } else {
    child.onCompleted();
    }
    return true;
    }
    return false;
    }

    If we detect unsubscription, we clear the queue and remove ourselves from the active set of subscribers (1). However, we don't need to call remove when reaching an done and empty state (2) because at this point, the state contains no subscribers anymore and is also terminated.

    A word about BehaviorSubject.

    BehaviorSubject is a special kind of subject sitting between a PublishSubject and ReplaySubject: it replays the very last onNext event before relaying other events then on. One might think it can be emulated via a size-bound ReplaySubject, however, their terminal behaviors differ. ReplaySubject of 1 will always replay 1 event and 1 terminal event whereas a terminated BehaviorSubject won't retain the very last onNext event and will emit only the terminal even.

    From concurrency perspective, the single value-replay creates complications when there is a race between a subscription and onNext. Since the requirement is that there can't be missed events from the point on where the subscription happens, one has to somehow capture the last value, emit it and then emit any other value.

    In the current 1.x implementation of BehaviorSubject, this is achieved via a per-subscriber lock and a split mode: first and next. When the subscription happens, the subscribing thread tries to enter into the first mode which reads out the last value from the subject and emits it. If there is a concurrent onNext call, it is temporarily blocked. Once the first mode finishes, it switches to next mode and now onNext calls are immediately relayed.


    protected void emitNext(Object n, final NotificationLite<T> nl) {
    if (!fastPath) {
    synchronized (this) {
    first = false;
    if (emitting) {
    if (queue == null) {
    queue = new ArrayList<Object>();
    }
    queue.add(n);
    return;
    }
    }
    fastPath = true;
    }
    nl.accept(actual, n);
    }

    protected void emitFirst(Object n, final NotificationLite<T> nl) {
    synchronized (this) {
    if (!first || emitting) {
    return;
    }
    first = false;
    emitting = n != null;
    }
    if (n != null) {
    emitLoop(null, n, nl);
    }
    }

    Basically, it is an asymmetric emitter loop: if the emitNext() wins, emitFirst()won't run and who is to say the first onNext() the child subscribes wasn't the last one when the subscription happens asynchronously?

    There is, however, a very subtle bug still lurking in this approach. It is possible emitFirst() will emit the same value twice!

    In the right interleaved conditions, onNext sets the last value in the state, emitFirst picks up the last state. Then onNext tries to run emitNext() which finds an emitting state and queues the value. Finally, emitFirst notices there is still work to do and dequeues the value and now we have emitted the same value twice.

    The solution, although works, is a bit complicated and can be seen here with RxJava 2.x. Basically one has to add a version tag to the value, lock out onNext() for a small duration and drop old indexed values when emitting. The clear drawback is that we now have another lock in the execution path and in theory, any concurrently subscribing child can now block the emitter thread. A lock-free approach is possible, but it requires allocation of an immutable value+index every time an onNext event appears.

    Conclusion

    In this post, I've shown how to implement a PublishSubject with 3 different kinds of backpressure-handling strategies. I consider this as the final piece about Subjects.

    If you look into RxJava 1.x, you may see that standard Subjects aren't implemented this way, however, 2.x Subjects are. This is no accident and the 2.x implementation come from lessons learned from the 1.x implementation.

    In the next blog post-series, we're going to utilize the knowledge about Subject internals and I'm going to show how to implement ConnectableObservables.

    Operator internals: introduction

    $
    0
    0
    Developing an operator is usually a non-trivial task. In the Advanced RxJava blog, I've tried to convey many of the foundational elements and experience I've got from building them.

    However, RxJava has around 150 unique operators and many of them required some custom logic or "unconventional" idea. Such knowledge is hard or next to impossible to explain in any meaningful generalization.

    Therefore, I'll start a parallel series where I'm going to dive into each operator, except perhaps the most trivial ones. I'll call all of them operators, regardless of whether they implement Operator or OnSubscribe for convenience. I'll go by name ascending and cover aliases of an operator into the same post.

    I'll look into both 1.x and 2.x implementations which has the added benefit of doing an effective review of them while also pointing out what it takes to make said operator Reactive-Streams compliant.



    Operator internals: Amb, AmbWith

    $
    0
    0

    Introduction


    The amb operator, shorthand for ambiguous, subscribes to a set of source Observables and keeps relaying events from the first Observable that signaled any event while unsubscribing and ignoring the rest. The operator can and does support backpressure.

    From the operator building's perspective, we need to consider the following properties/requirements:


    • The number of source Observables is known when the downstream subscribes.
    • We need to track all subscriptions in a collection other than CompositeSubscription because there is no way to cherry pick one and unsubscribe the rest.
    • One has to relay the downstream request to all sources.
    • Unsubscription, requesting and even the choosing of a "winner" may happen while subscribing to the source Observables.
    The two major versions are implemented slightly differently.

    Implementation: 1.x

    The 1.x implementation is slightly more verbose. It uses a simple ConcurrentLinkedQueue to keep track of the subscribers, AmbSubscriber. In addition, the winner AmbSubscriber is kept in an AtomicReference.

    In case an unsubscription happens, we need to attach a callback to the child subscriber which when called, will loop through the collection of AmbSubscribers and unsubscribes them one by one.


    When the subscription happens, a loop goes through all available sources, instantiated an AmbSubscriber and subscribes. Since unsubscription can happen at any time or any previous source may have already won, the loop has to check for both condition and quit early.


    Once all sources have been subscribed, the child subscriber receives its Producer. The task of this producer is to dispatch all requests to every AmbSubscriber, or in case of a winner, dispatch the request only to that particular AmbSubscriber.

    It may seem odd, but if there is a winner before the Producer is set, there is no need to set the Producer because the winner has obviously ignored any backpressure and started emitting anyways.

    In the AmbSubscriber, any event fired will check if the current AmbSubscriber is the winner or not. If so, the event is relayed. Otherwise, an atomic dance happens where the AmbSubscriber tries to CAS itself into the winning position and if successful, it unsubscribes the others. If the CAS failed, the AmbSubscriber unsubscribes itself.


    Implementation: 2.x

    The 2.x implementation is less verbose and exploits the fact that the number of source Observables is known. Therefore, an array of AmbInnerSubscriber is used and the "winner" indicator is now an volatile integer field backed by a field updater. 

    When the child subscribes, a loop first creates every AmbInnerSubscriber, sets a custom Subscription on the child (which is the coordinator class itself) and then subscribes to each source Observable. This second loop also checks for a winner in the process.

    The winner field has multiple meanings depending on the state of the operator. Minus one indicates the child cancelled, zero means there is no winner yet and any positive number indicates the index plus 1 of the winner AmbInnerSubscriber.

    In the reactive-streams world, there is an increased likelihood a Subscription arrives later than any request or cancellation attempt, therefore, one has to be prepared for it. Therefore, AmbInnerSubscriber has to keep its Subscription in a volatile field plus it has to track all the missed requests in another. This pattern is so common with 2.x, it is worth detailing it here:

    class AmbSubscriber<T> 
    extends AtomicReference<Subscription>
    implements Subscriber<T>, Subscription {
    volatile long missedRequested;
    static final AtomicLongFieldUpdater MISSED_REQUESTED = ...;

    static final Subscription CANCELLED = ...;
    }

    The class implements Subscriber, naturally, and Subscription for convenience (so we have request() and cancel() to implement). The class also has a static final field holding an empty implementation of the Subscription interface. We will use this instance to indicate a cancelled state and also notify any late-coming request() or onSubscribe() to do nothing. By extending AtomicReference directly, we will keep the incoming Subscription in a (hidden) instance field and access it via atomic methods of this.

    Let's see the implementation of the cancel() method first:


    @Override
    public void cancel() {
    Subscription s = get();
    if (s != CANCELLED) {
    s = getAndSet(CANCELLED);
    if (s != CANCELLED && s != null) {
    s.cancel();
    }
    }
    }

    This atomic getAndSet() should look familiar by now. When called, if the current subscription is not the constant CANCELLED, we getAndSet it to cancelled. The atomicity guarantees that there will be only one thread that experiences a non-CANCELLED previous state in which case we call cancel on it. Note that cancel() may be called before onSubscribe thus the current subscription may be null.

    Next, let's see the onSubscribe() method:


    @Override
    public void onSubscribe(Subscription s) {
    if (!compareAndSet(null, s)) { // (1)
    s.cancel(); // (2)
    if (get() != CANCELLED) { // (3)
    SubscriptionHelper.reportSubscriptionSet();
    }
    return;
    }

    long r = MISSED_REQUESTED.getAndSet(this, 0L); // (4)
    if (r != 0L) { // (5)
    s.request(r);
    }
    }


    1. First, we try to CAS in the incoming Subscription and replace a null value.
    2. If there is already a Subscription, we cancel the incoming one in any case.
    3. It is possible, although unlikely, multiple calls to onSubscribe happens due to bogous source. If the current value isn't the cancelled indicator, we have to report the incident in some way and just quit.
    4. If the CAS succeeded, we now have to take all missed requested amount via getAndSet.
    5. If there were in fact missed requests, we do request that amount from the Subscription at hand.


    Finally, let's look at the request() method:

    @Override
    public void request(long n) {
    Subscription s = get();
    if (s != null) { // (1)
    s.request(n);
    } else {
    BackpressureHelper.add(MISSED_REQUESTED, this, n); // (2)
    s = get();
    if (s != null && s != CANCELLED) { // (3)
    long r = MISSED_REQUESTED.getAndSet(this, 0L); // (4)
    if (r != 0L) { // (5)
    s.request(r);
    }
    }
    }
    }


    1. When the request is called, first we check if the current Subscription isn't null. If so, we request the amount directly. The current Subscription might be the CANCELLED instance in which case this call is a no-op.
    2. We use the backpressure-helper routine to safely add the number to the missedRequested field (which caps at Long.MAX_VALUE). 2.x Bug: validation of n is missing here.
    3. Once the missed amount has been added, we need to check the Subscription again since it might have been set asynchronously. 
    4. If not null and not cancelled, we call getAndSet the missed amount. This makes sure the missed amount is either retrieved by this method or by the onSubscribe method atomically.
    5. If the missed amount is non-zero, we request it from the Subscription. Otherwise, the onSubscribe has already taken any missed value for us.


    The other onXXX methods of the AmbInnerSubscriber work similarly to the 1.x version. There is a local won field (no need for volatile) that if set, serves as a fast-path for delivering events. If it is false, there is an attempt to win the race and if won, the won field is set to true. Otherwise the AmbInnerSubscriber cancels the Subscription at hand (which shouldn't be null at this point as RS requires calling onSubscribe before any other onXXX methods).

    2.x Bug: If the AmbSubscriber wins, it doesn't cancel the other AmbSubscribers and thus they remain subscribed indefinitely.

    Conclusion

    The operator amb isn't a complicated operator, 5/10 maybe, but it requires some custom logic to deal with unsubscription/cancellation and request dispatching.

    While reviewing the 2.x implementation, I found two oversights that can be easily addressed via a PR.




    Operator internals: All, Any, Exists

    $
    0
    0

    Introduction

    The operator all checks if a given condition (predicate) holds for all elements of the upstream, emitting a single true value at the end, or emits false immediately if the predicate returns false. The operator any is its logical inverse and looks quite like all, except it returns immediately if the predicate returns true and returns false for empty upstreams. They can and do support backpressure.

    We need to consider the following properties/requirements with this operator:


    • Since the output is a single value, one doesn't need to play around with request accounting and can let the operator request Long.MAX_VALUE from upstream. This gives the added benefit that it may trigger a fast-path and thus run with reduced overhead.
    • Since the output is a single value, emitted even if the upstream is empty, one has to prepare for handling request amounts from downstream and only then emit the result.

    Implementation: 1.x

    The 1.x implementation is straightforward. The Subscriber requests Long.MAX_VALUE and uses the SingleDelayedProducer to delay the emission of the resulting boolean until the downstream actually requests.

    Since backpressure handling is optional in 1.x, one can't emit false in case the predicate returns false in onNext because only the SingleDelayedProducer knows if there was actually an request call or not.


    Implementation: 2.x

    The 2.x implementation is a bit longer because I chose to inline the behavior of the SingleDelayedProducer and thus saving on allocation costs.

    The backpressure requirement still holds but with one exception: since the call to onSubscribe is mandatory, onNext is only ever called if there was a request to it. Therefore, the operator has to insert itself between the upstream and the downstream request-wise.

    Failing the predicate in onNext no longer requires buffering of the value but can be simply emit directly (because we know there was at least a request(1) beforehand). An empty upstream, however, still requires "buffering" the result until a request comes along. The related state machine is quite similar to the one described in an earlier post. The notable difference is that we know the delayed emission will always emit true thus no need for an instance variable holding it until needed.

    It is worth looking at the onNext() method in AllSubscriber:


    @Override
    public void onNext(T t) {
    if (done) { // (1)
    return;
    }
    boolean b;
    try {
    b = predicate.test(t);
    } catch (Throwable e) { // (2)
    lazySet(HAS_REQUEST_HAS_VALUE);
    done = true;
    s.cancel();
    actual.onError(e);
    return;
    }
    if (!b) {
    lazySet(HAS_REQUEST_HAS_VALUE); // (3)
    done = true;
    s.cancel();
    actual.onNext(false);
    actual.onComplete();
    }
    }


    1. Cancellation is best effort in both Reactive-Streams and 1.x Observables and one can't rely upon the cancellation alone. The done flag drops all events after the termination/cancellation of the operator.
    2. Callbacks can crash, in which case we set the state-machine to its terminal value HAS_REQUEST_HAS_VALUE which should prevent any value emission in a request call. In addition we set the done flag and call cancel on the Subscription.
    3. If the predicate returned false, we can shortcut the stream by cancelling it and emitting the constant false as the result. Here, the state machine is also brought to its terminal state.


    Conclusion

    The all and any operators is among the simpler operators, 2/10 maybe, but one needs to recognize an empty upstream would overflow the downstream in a naive implementation and thus there is a need for the SingleDelayedProducer to bridge the gap.



    ConnectableObservables (part 1)

    $
    0
    0

    Introduction


    We learned about constructing cold (i.e., range) and hot observables (i.e., UnicastSubject) but nothing specific so far about how to convert between the two.

    Clearly, since subjects are also Observers, one only has to subscribe them to a cold source and let all the child Subscribers subscribe to the subject only.

    But why would one do that in the first place? The conversion has one major benefit, namely it makes side-effects in the cold source happen once (per Subject subscribed to it). From a usage perspective, it means that you can reuse the same stream for multiple purposes and not having multiple and likely independent sequences.

    For example, if you wanted to work on subsequent elements of the same stream, you could publish it and observe different parts of it through different subscriptions and combine the results:


    Observable<Integer> source = Observable.range(1, 10);

    ConnectableObservable<Integer> published = source.publish();

    Observable<Integer> first = published;
    Observable<Integer> second = published.skip(1);

    Observable<String> both = first.zipWith(second,
    (a, b) -> a + "+" + b);

    both.subscribe(System.out::println);

    published.connect();


    Now let's see what ConnectableObservables should do.


    ConnectableObservable requirements

    ConnectableObservable is an abstract class that extends Observable and requires one extra method to be implemented.

    By extending an Observable, it is subject to the same construction difficulties as are Subjects: namely their constructor requires an OnSubscribe callback which can't really access the outer class' methods at construction time so one needs a factory and an intermediate state object.

    The second requirement, also coming from Observable, is that subscription should be thread safe and the implementation should allow it to happen any time, before, during and after the ConnectableObservable"runs".

    It may come as a surprise that the extra abstract method isn't connect() but connect(Action1<Subscription> s) instead. The reason for this is due to the synchronous unsubscription possibility with a ConnectableObservable. But when does this come into play?

    There are two cases when this feature is essential, one is more public and one is hidden away in certain operators.

    The problem with connect() is that if it connects to an underlying cold and synchronous observable, that could run to completion (or never terminate) thus the method never returns. If you didn't have subscribers subscribed to it then those values may be gone forever. In addition, given an infinite synchronous stream, you may attempt to unsubscribe it via the Subscription returned by connect() after a while but then again, connect() never returns.

    This comes up quite often with the second case mostly with multicasting operator overloads such as publish(Func1) and replay(Func1). These operators create a ConnectableObservable behind the scenes, run it through the Func1 provided and return a plain Observable which when subscribed to will connect the ConnectableObservable. Now if the source is synchronous and you want to take only a few elements of the returned Observable, the child subscription would never return a Subscription and the whole stream would just keep running.

    The solution is to have the second, callback version of connect implemented which calls the callback with a subscription before it connects and thus allows it to be unsubscribed in sequence.

    Finally, connection and disconnection has to be idempotent. It means that calling connect twice on a running stream should do nothing as well as calling unsubscribe on such stream twice should unsubscribe a running stream once. One extra thing to be careful with unsubscription is that if one unsubscribes a stream the connects again, the Subscription from the first connection should not affect the state of the second connection.

    To summarize, ConnectableObservable has to

    • be thread safe when subscribing to it at any time and from any thread,
    • allow synchronous unsubscription at any time and from any thread and
    • be idempotent in respect of connect and disconnect (unsubscribe).

    A basic implementation

    Given what we know about ConnectableObservables and Subjects so far, it may come trivial to implement the former with the help of the latter. Let's implement a ConnectableObservable which takes a subject of your chosing and "publishes" a source Observable's values through it.

    public final class Multicast<T>
    extends ConnectableObservable<T> {

    final Observable<T> source;
    final Subject<T, T> subject;

    final AtomicReference<Subscription> subscription; // (1)

    public Multicast(Observable<T> source,
    Subject<T, T> subject) {
    super(s -> {
    subject.subscribe(s); // (2)
    });
    this.source = source;
    this.subject = subject;
    this.subscription = new AtomicReference<>();
    }

    @Override
    public void connect(
    Action1<? super Subscription> connection) {
    // implement
    }
    }

    So far, nothing special. We take a source observable, a subject and we will hold the current connection in an AtomicReference instance (1). The OnSubscribe logic is this case is simple and there is no need for the factory approach unlike UnicastSubject: for each incoming subscriber, we subscribe them to the subject directly (2).

    The body of the connect() method is a bit more involved but not too complicated:


    @Override
    public void connect(Action1<? super Subscription> connection) {
    for (;;) {
    Subscription s = subscription.get(); // (1)
    if (s != null) {
    connection.call(s); // (2)
    return;
    }

    Subscriber<T> subscriber = new Subscriber<T>() { // (3)
    @Override
    public void onNext(T t) {
    subject.onNext(t);
    }

    @Override
    public void onError(Throwable e) {
    subject.onError(e);
    }

    @Override
    public void onCompleted() {
    subject.onCompleted();
    }
    };

    subscriber.add(Subscriptions.create(() -> { // (4)
    subscription.set(null);
    }));

    if (subscription.compareAndSet(null, subscriber)) { // (5)
    connection.call(subscriber); // (6)

    source.subscribe(subscriber); // (7)

    return;
    }
    }
    }

    The implementation is basically a CAS loop:

    1. We keep the current connection's Subscription in the subscription field and if it is not null, it means there is an active connection.
    2. Given an active connection, we simply call the action with it.
    3. Otherwise, there seems to be no active connection and we have to establish one. You may think, why not subscribe the subject directly to the source? The reason is the requirement of synchronous unsubscription: the call to subscribe() returns a Subscription too late, thus we need a Subscription before that. The Subscriber we create will forward events and also present this unsubscription possibility (remember, Subscriber extends Subscription).
    4. When the connection, our Subscriber is unsubscribed, we have to set the subscription field back to null, allowing the next connect() to happen.
    5. To achieve idempotence with a connect, we CAS in a subscriber in place of a null value. If it fails, due to a concurrent call to connect(), the loop resumes at (1).
    6. If the CAS succeeded, we first call the callback with our Subscriber which will allow synchronous cancellation of the connection.
    7. Finally, we subscribe our Subscriber to the source and quit.

    Limitations of the basic implementation

    The basic implementation seem to work but has some limitations.

    Side note: With this blog, I hope to teach the reader how to detect bugs and shortcomings in operators; this is why some examples are not prepared for everything up front.

    The first limitation is that if the source terminates, the clearing of the subscription may happen sometime in the future (through a SafeSubscriber) or not at all. The solution is to clear the subscription in the onError and onCompleted methods of our Subscriber, but we can't use set(null) there. We have to conditionally clear it there and in the regular unsubscription path because it is possible that subscription is cleared by the other party (termination vs. unsubscription race). In short, the methods should be changed like this:


        // ...
    @Override
    public void onError(Throwable e) {
    subject.onError(e);
    subscription.compareAndSet(this, null);
    }

    @Override
    public void onCompleted() {
    subject.onCompleted();
    subscription.compareAndSet(this, null);
    }
    // ...

    subscriber.add(Subscriptions.create(() -> {
    subscription.compareAndSet(subscriber, null);
    }));

    In all three places, the clearing only happens if the current connection is still the known subscriber. This way, if there is a termination by any means followed by a reconnection, an unsubscribe() call to an old connection won't affect the new connection.

    The second limitation is that once the source runs to termination, the Subject will come to its terminal state as well. New connection attempts will disconnect immediately and child Subscribers will only receive a terminal event (with the standards Subjects of RxJava).

    Most likely this isn't what the business logic dictates, therefore, we have to change the parametrization of the Multicast so we can get a fresh Subject for any new connection. I'll show an implementation of this in the next subsection but before that, let's see the final limitation of the basic implementation.

    The final limitation is that there is no request coordination: our Subscriber and the Subject itself will run in unbounded mode and ignore all backpressure requests. Since the standard RxJava 1.x Subjects don't support backpressure, we may run into MissingBackpressureExceptions somewhere in the downstream. Although 2.x Subjects are backpressure-aware, 2.x PublishSubject will still throw MissingBackpressureException if the child subscriber can't keep up and 2.x ReplaySubject does effectively unbounded buffering (similar to onBackpressureBuffer)

    The resolution is a larger step up on the complexity ladder and will be detailed in the next part of this series about ConnectableObservables.


    Fresh Subject on connect

    The solution to the lack of reusability with the basic implementation can be solved by using a supplier function instead of a Subject instance and call it just before the connection happens.

    This, however, creates another problem. Because the subject doesn't exist the time the constructor sets the OnSubscribe callback, we somehow have to remember the Subscribers that have attempted to subscribe when there was no connection yet but then subscribe to the actual Subject when there is a connection.

    First, we now have to manage a more complex state. I'll create a Connection class that represents the state of a connection:


    static final class Connection<T> {
    Subject<T, T> subject; // (1)
    List<Subscriber<? super T>> subscribers; // (2)
    boolean connect; // (3)
    final SerialSubscription parent; // (4)

    public Connection() {
    this.subscribers = new ArrayList<>();
    this.parent = new SerialSubscription();
    }

    public void setSubject(Subject<T, T> subject) { // (5)
    // implement

    }

    public void subscribe(Subscriber<? super T> s) { // (6)
    // implement
    }

    public boolean tryConnect() { // (7)
    // implement
    }
    }

    Let's see its parts:


    1. We have to store a Subject so subscribers can be subscribed to it any time.
    2. Since the subject doesn't exist until connect() is called, we have to store the early birds in a list and subscribe them all once the subject becomes available.
    3. The connection has to happen once per Connection object (termination or unsubscription then has to create an entirely new Connection object, see later).
    4. We have to keep reference to the subscription to the source Observable. However, we can't just store a Subscriber because the connection process may longer at which a concurrent connection might found that reference to be still null (unlike the basic example where the Subscription was atomically established). The container is non null and ensures proper unsubscription on arrival if necessary.
    5. We have to set a Subject once available and subscribe all early bird Subscribers to it.
    6. We also have to provide a way for the OnSubscribe in the constructor to add new subscribers properly, depending on the current state of the connection.
    7. Finally, connection has to happen once per Connection object which is managed by the tryConnect() method.

    The implementation of the methods (5-7) are relatively simple but need some short explanation:

    public void setSubject(Subject<T, T> subject) {
    List<Subscriber<? super T>> list;
    synchronized (this) {
    this.subject = subject;
    list = subscribers;
    subscribers = null;
    }
    for (Subscriber<? super T> s : list) {
    subject.subscribe(s);
    }
    }

    In this method, the subject is set while holding a lock on this. The reason for it is that to prevent concurrent subscribe() calls (see below) to happen while the Subject is set. This way, early bird Subscribers will be subscribed to the subject in this method whereas late subscribers will be directly subscribed to the subject once the unlock happens, skipping the list entirely. Subscribing the early birds outside the lock reduces the likelihood of deadlock and also doesn't block the concurrent subscribers while the loop is running.

    Next comes the subscribe() method.


    public void subscribe(Subscriber<? super T> s) {
    Subject<T, T> subject;
    synchronized (this) {
    subject = this.subject;
    if (subject == null) {
    subscribers.add(s);
    return;
    }
    }
    subject.subscribe(s);
    }

    What happens here is that, atomically, if the subject is still null (i.e., connect() hasn't been called yet), we add the subscriber to the inner list. If, however, the subject is non-null, we subscribe the Subscriber directly to it. One optimization here would be to have the subject field volatile and do a double-checked locking (since subject will be only set once).

    Finally, the tryConnect() method is pretty simple:

    public boolean tryConnect() {
    synchronized (this) {
    if (!connect) {
    connect = true;
    return true;
    }
    return false;
    }
    }

    Atomically check if the connection is false and switch it to true. If this switch happened, return true, otherwise return false. The former will trigger the connection logic while the latter will simply "return" the parent SerialSubscription in connect() (detailed later).

    I'll call the new ConnectableObservableMulticastSupplier and it will have the following skeleton:


    public final class MulticastSupplier<T> 
    extends ConnectableObservable<T> {
    public static <T> MulticastSupplier<T> create( // (1)
    Observable<T> source,
    Supplier<Subject<T, T>> subjectSupplier) {
    AtomicReference<Connection<T>> conn =
    new AtomicReference<>(new Connection<>()); // (2)

    return new MulticastSupplier<>(
    source, subjectSupplier, conn);
    }



    final Observable<T> source;
    final Supplier<Subject<T, T>> subjectSupplier;
    final AtomicReference<Connection<T>> connection; // (3)


    protected MulticastSupplier(Observable<T> source,
    Supplier<Subject<T, T>> subjectSupplier,
    AtomicReference<Connection<T>> connection) {
    super(s -> {
    Connection<T> conn = connection.get(); // (4)
    conn.subscribe(s);
    });
    this.source = source;
    this.subjectSupplier = subjectSupplier;
    this.connection = connection;
    }

    void replaceConnection(Connection<T> conn) { // (5)
    Connection<T> next = new Connection<>();
    connection.compareAndSet(conn, next);
    }

    @Override
    public void connect(Action1<? super Subscription> connection) {
    // implement
    }
    }

    Let's see why this looks like as it is:

    1. Since the constructor doesn't allow access to instance field before super is called, we have to create the Connection state before the instantiation of MulticastSupplier so both its body and the OnSubscribe callback can access it. 
    2. Since the connection is not constant (can be reconnected any number of times), we have to use a holder for the connection, an AtomicReference in this case. It will come in handy when the state changes have to happen atomically. In addition, since Subscribers have to be remembered before a connection is established, we can't use a null state anymore.
    3. The same AtomicReference has to be accessible from connect() later on.
    4. The OnSubscribe callback is slightly modified: we retrieve the current Connection instance and call subscribe() on it. If it is connected, it will go straight and subscribe to the underlying Subject, otherwise the Subscriber will be remembered.
    5. Finally, if either a disconnect or source termination happens, we have to replace the old connection with a fresh one so connect() can start again. The implementation first creates a new (empty) Connection and tries to CAS it in, replacing the known Connection when the last connect() has established it. This will prevent old Subscriptions to disconnect newer connections.
    Finally, let's see the connect() implementation:


    @Override
    public void connect(
    Action1<? super Subscription> connection) {

    Connection<T> conn = this.connection.get(); // (1)

    if (conn.tryConnect()) { // (2)
    Subject<T, T> subject = subjectSupplier.get();

    Subscriber<T> parent = new Subscriber<T>() { // (3)
    @Override
    public void onNext(T t) {
    subject.onNext(t);
    }

    @Override
    public void onError(Throwable e) { // (4)
    subject.onError(e);
    replaceConnection(conn);
    }

    @Override
    public void onCompleted() {
    subject.onCompleted();
    replaceConnection(conn);
    }
    };

    conn.parent.set(parent); // (5)

    parent.add(Subscriptions.create(() -> { // (6)
    replaceConnection(conn);
    }));

    conn.setSubject(subject); // (7)

    connection.call(conn.parent); // (8)

    source.subscribe(parent); // (9)
    } else {
    connection.call(conn.parent); // (10)
    }
    }

    The implementation no longer has a CAS loop because the atomic connection requirement is handled a bit differently:


    1. First, we retrieve the current Connection object from the AtomicReference.
    2. If not connected, set the state to connected and perform the connection logic, otherwise, go to (10).
    3. Once we got a Subject, we create our wrapper Subscriber as before that references the Subject.
    4. We'd like to disconnect eagerly once a terminal event has been received, therefore, we call replaceConnection() with the known connection object. Depending on what kind of races one wish to tolerate, you can swap the call on the subject with the replacement: this way, subscribers racing with the termination event will be added to the next connection instead of the current.
    5. We then set the parent onto the connection. If there was a concurrent disconnect, this will unsubscribe the parent Subscriber immediately. Depending on how a connect-disconnect race should be handled, one can quit if isUnsubscribed() is true, don't even try to subscribe to the source but return a unsubscribed Subscription or retry the connection attempt. The latter requires a similar loop as in the basic example.
    6. We set the unsubscribe action to replace the connection.
    7. We set the subject on the current connection, which may trigger the early birds' subscription to the Subject.
    8. Before the parent Subscriber is connected, we call the callback with the SerialSubscription (not the Subscriber!) to allow synchronous cancellation.
    9. Then we subscribe the parent Subscriber to the source Observable.
    10. If the tryConnect() returned false, we simply call the callback with the SerialSubscription of the current connection. Note here that this has to be non-null and thus the need of indirection around the parent Subscriber of (3).


    Conclusion

    In this blog post, I've detailed the requirements of ConnectableObservables and showed two simpler variants of implementing one.

    However, one would expect request coordination from a ConnectableObservable which neither of the Multicast or MulticastSelector supports.

    Looking at them wasn't in vain, because they feature construction approaches that will come in handy with the next part of this mini-series.

    So far, operators and classes were low to medium complexity due to the fact that the event and method call "streams" were not really stepping on each other. Next, however, we step up on the complexity ladder and look at how one can coordinate requests within a ConnectableObservable.

    This is, in my opinion, a master-level implementation task and if understood, it opens the door to the most complex operator implementations in RxJava. Stay tuned!


    Operator internals: AutoConnect

    $
    0
    0

    Introduction


    The operator autoConnect is a member of the ConnectableObservable class and allows triggering the connection to the underlying ConnectableObservable once the specified amount of subscribers have arrived. The operator returns a plain Observable and as such can be more easily included in a chain of operators.

    There are two reasons why this operator exists. First, many wanted to connect to a ConnectableObservable only if a given number of subscribers have subscribed to it which, before that, was difficult to achieve due to the lack of confinement. The second reason was that another operator, cache, didn't support advanced retention policies such as size and/or time bounds and it was somewhat tedious to achieve the same first-subscriber triggered connection as cache does.

    To sketch its implementation, one needs an AtomicInteger to count each subscriber in an OnSubscribe callback and once the count reaches the desired amount, the call to connect() can happen.

    There is, however, a small complication: the synchronous unsubscription support in ConnectableObservable. By applying plain autoConnect, one loses the means to unsubscribe an ongoing stream, similar to how cache behaves. The resolution is to take a callback and hand it to the connect() method.

    Implementation details

    The operator is not involved in request management at all, therefore, the implementation in 1.x and the two implementation in 2.x (the other is on the NbpConnectableObservable) looks essentially the same.

    In fact, it is so short I'm going to repeat it here:

    public Observable<T> autoConnect(int numConnections,
    Action1<Subscription> connection) { // (1)
    if (numConnection == 0) {
    connect(connection); // (2)
    return this;
    }
    AtomicInteger count = new AtomicInteger();
    return create(s -> {
    unsafeSubscribe(s); // (3)
    if (count.incrementAndGet() == numConnections) { // (4)
    connect(connection);
    }
    });
    }

    Let's discuss the interesting points:

    1. RxJava has two extra overloads of this method, one that defaults to 1 required connection and the other asks for the number of connections. Both ignore the connection callback.
    2. If the number of connection is zero, we interpret it as an immediate connection. In this case, we don't have to do any kind of wrapping and just return the ConnectableObservable instance as is.
    3. If the number of connection is non-zero, we have to capture the subscription attempts and do extra work once the number of subscribers reached the required amount. Before we even test for that, we subscribe the incoming Subscriber to the underlying ConnectableObservable. This is necessary to happen first because if the numConnections is 1, the connection may drain the underlying sequence and the Subscriber may not receive any values at all.
    4. Once the subscriber count reached the required amount, we trigger a connection (that will call back the connection callback supplied originally.
    At this point, you might think what happens if numConnections is 2, a Subscriber subscribes then unsubscribes immediately. Should the next Subscriber really trigger the connection? It depends on your requirements. The autoConnect operator, clearly, doesn't do this (a decrementAndGet() somewhere would indicate this). 

    One reason for this is that originally, the operator was meant to be a simple replacement for using refCount() or share() in certain situations.


    Conclusion

    The operator autoConnect is among the simplest operators there are, 1 / 10, and thus has clear and simple feature set.


    ConnectableObservables (part 2)

    $
    0
    0

    Introduction


    In the previous post, I've shown how one can write a "simple"ConnectableObservable that uses a Subject to dispatch events to subscribers once it has been connected.

    The shortcoming of the solution is that there is no request coordination and everything runs in unbounded mode: the developers have to apply onBackpressureXXX strategies per subscriber, however, that leads to either dropping data or buffer bloat.

    If the underlying Observable is cold, there should be a way to make sure it emits only as much elements as the child subscribers can process. To achieve this, we need request coordination.


    Request coordination

    So far, the operators we were implementing had to deal with a single child subscriber and its request at a time. One had to either pass it through, rebatch it or accumulate it, based on the business logic of said operator.

    When there are multiple child Subscribers, the problem space suddenly receives a new dimension. What are the new problems?

    Every bit counts

    First, different child subscribers may request different amounts. Some may request small amounts, some may request larger amounts and others may want to run in unbounded mode (i.e., request(Long.MAX_VALUE)). In addition, the request calls may happen any time and with any amount.

    Given such heterogeneous request pattern, what should be the request amount sent to the upstream Observable source?

    There are two main options:

    1. request as much that the smallest child Subscriber requested and
    2. request as much as the largest child Subscriber requested.
    Option 1) is essentially the lockstep approach. Its benefit is that there is no no need for request re-batching and buffering since once the upstream emits, everybody can receive it immediately. (Rebatching and buffering is an option in case the request amounts are really 1s or 10s at a time.) The drawback is that the whole setup slows down to the slowest child Subscriber, which if "forgets" to request, nobody gets anything.

    Option 2) gives more room to individual child Subscribers and allows them to run on their own pace. However, this solution requires unbounded buffering capability (which may be shared or per each Subscriber). This means if there is an unbounded child Subscriber, the operator has to request Long.MAX_VALUE and fill the buffers for everyone. This, depending on the operator, may be of no problem though.

    Subscribers may come and go at will

    The second problem is that the the number of Subscribers may not be constant: new subscribers arrive, old ones leave. This poses another set of problems:

    1. A child Subscriber may request Long.MAX_VALUE then leave after a few (or no) elements.
    2. A child Subscriber may arrive but not request anything, stopping everyone else.
    3. A child Subscriber may leave at any time and thus its request amount "pressure" has to be released.
    4. All child Subscribers leave before the upstream Observable completes. What should happen in this case?
    Unfortunately, problems 1) and 2) require mutually exclusive approaches explained above (lockstep vs. unbounded buffering). Problem 3) requires unsubscription action.

    Problem 4) depends on the approach taken in respect to 1) and 2).

    Within the lockstep approach, two sub-options arise. Either one has to introduce some bounded buffers that will hold onto the requested amounts, which now has to be re-batched to fit in, and simply await the new Subscribers. Otherwise, one has to slowly "drip" away the source values until a child Subscriber arrives. 

    Within the unbounded buffering approach, one can simply keep buffering or again, start dropping values.

    Approaches taken in RxJava

    RxJava has two operators that return a ConnectableObservable: publish() and replay(). For a long time, these were ignoring backpressure completely and behaved just like the MulticastSupplier in the previous part.

    These operators were rewritten to support backpressure (in 1.0.13 and 1.0.14 respectively) and had to take the problems mentioned before into account. The solutions were as follows

    Operator publish() does lockstepping with a fixed prefetch buffer: the buffer is only drained (and then replenished) if all known child Subscribers can take a value. If there are no child Subscribers, it "slowly drips away" it source, which means it starts to request 1 by 1 and drops these values.

    Operator replay() does unbounded buffering. The reason for this is that both the bounded and unbounded version of replay() has to buffer and replay all values from the upstream anyway. You may think, why buffer everything when the replay is time and/or size bound. The answer is that these operators, similar to Subjects, have to deliver events continuously and without skips; if there is an child Subscriber that arrived at some time, requested 1 then went to "sleep", the next time it requests the bounded replay has to present the next value, no matter how far ahead the other Subscribers went in the meantime.


    The effect of disconnection

    There is a problem that isn't dealt with in the RxJava operators but has to be mentioned. If one unsubscribes the Subscription returned by the connect() method, the upstream will stop sending further events.

    The problem is that this may leave the child Subscribers hanging: they won't receive any further events (beyond those that are already in some buffer of the respective operator). We have similar problems with CompletableFutures in Java 8. One can cancel a Future but what happens to those that were awaiting its result?

    The solution in Java 8 is to emit a CancellationException as the result in this case so that the dependent computations can terminate. However, this isn't the case with RxJava (in both 1.x and 2.x branches). The current implementation will just hang the child Subscribers.

    This problem may appear outside of a ConnectableObservable as well. For some time, the RxAndroid 0.x library contained an operator that were applied to all sequences and unsubscribed them if the lifecycle required cleanup. The problem was that this left child Subscribers without termination events. I suggested emitting an onError and onCompleted event for this case. There was no resolution of the problem and the operator was removed before 1.0.

    On a personal note, I don't remember anyone from the community complaining about this problem and it seems nobody is really affected by this behavior. As with many obscure and corner cases, if I don't mention them, nobody else seems to discover them.

    The effect of termination

    Upstream Observables may terminate normally, in which case the ConnectableObservable will emit the terminal event to child Subscribers.

    At this point, a new Subscriber may subscribe to the terminated ConnectableObservable. What should happen in this case? Does the termination also mean disconnection? Should the child Subscriber get terminated instantly, similar to PublishSubject?

    Again the solution requires business decision. RxJava chose the approach that a terminal event sent to a ConnectableObservables is considered a disconnect event and late coming Subscribers won't receive any terminal event but will be remembered until another call to connect() happens.

    This has the benefit that the developers can "prepare" child Subscribers before the upstream Observable gets run and thus avoid losing events. The drawback is that one has to remember to call connect() again, otherwise nothing runs and the Subscribers are left hanging.

    Family of collectors and emitters

    Before we jump into some code, I'd like to sketch out a pattern that is the foundation of almost all operators that deal with either multiple sources or multiple child Subscribers.

    I've written dozens of such operators and I've noticed they all use the same set of components and methods:

    1. They all need to track Subscribers, either the child Subscribers or the Subscribers that are subscribed to the source Observables. The tracking structure uses the copy-on-write approach of array-based resource containers.
    2. They all use an emitter loop (synchronized) or drain loop (atomics) which has to be triggered from many places: when an event is emitted from upstream(s), when a new child Subscriber arrives, when a request comes from child Subscribers and sometimes when a child unsubscribes.
    3. The loop has some preprocessing step: figuring out where the Subscribers are at the moment, selecting which source to drain or combining available values from sources in some fashion
    4. Finally, the events are delivered to Subscriber(s) and replenishments are requested from source Observable(s).

    Which operator?

    Now that we are aware of the problems, let's implement a ConnectableObservable which does request coordination.

    I've been thinking what operator to implement. My first thought was to show how to implement the operator pair of an AsyncSubject or BehaviorSubject (similar to how publish() is the pair of PublishSubject), however, the former can be implemented using plain composition plus replay():

    public ConnectableObservable<T> async() {
    return takeLast(1).replay();
    }

    Implementing the pair of BehaviorSubject is a bit more involved. The naive implementation would use composition such as this:

    public ConnectableObservable<T> behave() {
    return replay(1);
    }

    However, this doesn't properly capture the behavior of a terminated BehaviorSubject: child Subscribers get nothing but a terminal event whereas replay will always replay 1 value and 1 terminal event after it completed.

    To minimize brain melting, I'm not going to show how to implement a variant of the least complex of the operators: publish().


    Publish (or die)

    First, let's sketch out all the requirements we want to achieve:


    1. The operator should do a lockstep-based request coordination with prefetching (for efficiency)
    2. The effect of disconnection on the child Subscribers should be parametrizable: no event, signal error or signal completion.
    3. The operator should be considered terminated and new subscribers will wait for the next connect().
    4. The operator will allow errors to cut ahead. (Implementing error-delay is an excercise left to the reader).
    5. The operator will use a power-of-2 prefetch buffer.


    With these requirements, we start with the skeleton of the class as usual:


    public class PublishConnectableObservable<T> 
    extends ConnectableObservable<T> {

    public enum DisconnectStrategy { // (1)
    NO_EVENT,
    SEND_ERROR,
    SEND_COMPLETED
    }

    public static <T> PublishConnectableObservable<T> 
    createWith( // (2)
    Observable<T> source,
    DisconnectStrategy strategy) {
    State<T> state = new State<>(strategy, source);
    return new PublishConnectableObservable<>(state);
    }

    final State<T> state;

    protected PublishConnectableObservable(State<T> state) { // (3)
    super(state);
    this.state = state;
    }

    @Override
    public void connect(
    Action1<? super Subscription> connection) { // (4)
    state.connect(connection);
    }
    }

    Nothing extraordinary so far:

    1. We create an enum for the disconnection strategy
    2. We have to use a factory method because the internal state has to be accessible from OnSubscribe and from instance methods of this class.
    3. We construct the object where State doubles as an OnSubscribe to save on allocation.
    4. Finally, we delegate the connection attempt to the state object. This gives us a less verbose source code.
    Next comes the state object with some familiar structure (see last post of this series):

    static final class State<T> implements OnSubscribe<T> {
    final DisconnectStrategy strategy;
    final Observable<T> source;

    final AtomicReference<Connection<T>> connection; // (1)

    public State(DisconnectStrategy strategy,
    Observable<T> source) { // (2)
    this.strategy = strategy;
    this.source = source;
    this.connection = new AtomicReference<>(
    new Connection<>(this)
    );
    }

    @Override
    public void call(Subscriber<? super T> s) { // (3)
    // implement
    }

    public void connect(
    Action1<? super Subscription> disconnect) { // (4)
    // implement
    }

    public void replaceConnection(Connection<T> conn) { // (5)
    Connection<T> next = new Connection<>(this);
    connection.compareAndSet(conn, next);
    }
    }

    The state object will handle the connection, subscription and reconnection cases:

    1. Because we have to reconnect, we store the current connection in an AtomicReference.
    2. We initialize the source and strategy fields and set up an initial unconnected connection.
    3. The method call() from OnSubscribe will handle the subscribers; I'll show the implementation further down.
    4. The connect method will handle the connection attempts; I'll show the implementation further down.
    5. Finally, once a connection has been terminated on its own or via unsubscribe, we have to replace the old connection with a fresh connection atomically and not overwriting somebody else's fresh connection due to races.
    Before going deep into the complicated logic, two more simplistic classes remain. The first is the Subscriber that will be subscribed to the source Observable:


    static final class SourceSubscriber<T> 
    extends Subscriber<T> {
    final Connection<T> connection;
    public SourceSubscriber(
    Connection<T> connection) { // (1)
    this.connection = connection;
    }
    @Override
    public void onStart() {
    request(RxRingBuffer.SIZE); // (2)
    }

    @Override
    public void onNext(T t) {
    connection.onNext(t); // (3)
    }

    @Override
    public void onError(Throwable e) {
    connection.onError(e);
    }

    @Override
    public void onCompleted() {
    connection.onCompleted();
    }

    public void requestMore(long n) { // (4)
    request(n);
    }
    }

    The class, again is full of delegations:

    1. We store the connection object and we will delegate events to it.
    2. If this Subscriber is subscribed to the source Observable, we request only a limited number of elements upfront. (Parametrizing this is left to the reader).
    3. Again, for class simplicity, we delegate the events to the connection object, which happens to implement the Observer interface for convenience
    4. We will have to replenish all consumed values but request() is a protected method: it is exposed through the requestMore() method.
    Next comes a Publisher and Subscriber instance that will handle the unsubscription and request accounting for the child Subscribers of our operator.

    static final class PublishProducer<T> 
    implements Producer, Subscription {
    final Subscriber<? super T> actual;
    final AtomicLong requested;
    final AtomicBoolean once;
    volatile Connection<T> connection; // (1)

    public PublishProducer(
    Subscriber<? super T> actual) {
    this.actual = actual;
    this.requested = new AtomicLong();
    this.once = new AtomicBoolean();
    }

    @Override
    public void request(long n) {
    if (n < 0) {
    throw new IllegalArgumentException();
    }
    if (n > 0) {
    BackpressureUtils
    .getAndAddRequest(requested, n);
    Connection<T> conn = connection; // (2)
    if (conn != null) {
    conn.drain();
    }
    }
    }

    @Override
    public boolean isUnsubscribed() {
    return once.get();
    }

    @Override
    public void unsubscribe() {
    if (once.compareAndSet(false, true)) {
    Connection<T> conn = connection; // (3)
    if (conn != null) {
    conn.remove(this);
    conn.drain();
    }
    }
    }
    }

    This is a bit more interesting.

    1. We need to know about what connection this class has to deal with for two reasons: 1) it has to notify the connection the underlying Subscriber can receive values, 2) if the subscriber unsubscribes, it may mean the other Subscribers can now receive further values.
    2. Since request() runs asynchronously, the connection might not be available yet. We have to remember to call drain() once this connection becomes available (shown later on).
    3. Since unsubscribe() runs asynchronously as well, it has check for non-null and only remove itself from the array of subscribers (shown later on). Note also the idempotence provided by once.

    The final class, in skeleton form is the Connection itself:


    @SuppressWarnings({ "unchecked", "rawtypes" })
    static final class Connection<T>
    implements Observer<T> { // (1)

    final AtomicReference<PublishProducer<T>[]>
    subscribers;
    final State<T> state;
    final AtomicBoolean connected;

    final Queue<T> queue;
    final AtomicReference<Throwable> error;
    volatile boolean done;

    volatile boolean disconnected;

    final AtomicInteger wip;

    final SourceSubscriber parent;


    static final PublishProducer[] EMPTY =
    new PublishProducer[0];

    static final PublishProducer[] TERMINATED =
    new PublishProducer[0];

    public Connection(State<T> state) { // (2)
    this.state = state;
    this.subscribers = new AtomicReference<>(EMPTY);
    this.connected = new AtomicBoolean();
    this.queue = new SpscArrayQueue(
    RxRingBuffer.SIZE);
    this.error = new AtomicReference<>();
    this.wip = new AtomicInteger();
    this.parent = createParent();
    }

    SourceSubscriber createParent() { // (3)
    // implement
    }

    boolean add(PublishProducer<T> producer) { // (4)
    // implement
    }

    void remove(PublishProducer<T> producer) {
    // implement
    }

    void onConnect(
    Action1<? super Subscription> disconnect) { // (5)
    // implement
    }

    @Override
    public void onNext(T t) { // (6)
    // implement
    }

    @Override
    public void onError(Throwable e) {
    // implement
    }

    @Override
    public void onCompleted() {
    // implement
    }

    void drain() { // (7)
    // implement
    }

    boolean checkTerminated(boolean d,
    boolean empty) {
    // implement
    }
    }

    The method names and fields should look familiar by now:


    1. The class has to manage a set of state variables: the current array of Subscribers, the value queue plus the terminal event holders, the connection and disconnection indicators, the work counter for the queue-drain approach, the Subscriber that is subscribed to the Observable and finally the EMPTY and TERMINATED array indicators.
    2. The constructor initializes the various fields.
    3. The subscriber needs some preparations besides creating a new SourceSubscriber, therefore, I factored it out into a separate method.
    4. The copy-on-write handling of the known subscribers is done via add and remove, similar to how we did this with Subjects and with the array-backedSubscription container.
    5. We will handle the source events with these onXXX methods.
    6. Finally, the drain and termination check methods for the queue-drain approach.


    The meltdown

    So far, the classes and those methods implemented were nothing special. However, the real complexity starts from here on. I'll show the missing implementations one by one and mention the concurrency considerations with them as well..

    I suggest you take a small break, drink some power-up, clear your head at this point.

    Done? All right, let'd do this.

    State.call

    This method is responsible for handling the incoming child Subscribers. The method has to consider that the connection may terminate on its own or get disconnected concurrently:


    @Override
    public void call(Subscriber<? super T> s) {
    PublishProducer<T> pp
    = new PublishProducer<>(s);

    s.add(pp);
    s.setProducer(pp); // (1)

    for (;;) {
    Connection<T> curr = connection.get();

    pp.connection = curr; // (2)
    if (curr.add(pp)) { // (3)
    if (pp.isUnsubscribed()) { // (4)
    curr.remove(pp);
    } else {
    curr.drain(); // (5)
    }
    break;
    }
    }
    }


    1. First, we create a PublishProducer and set it on the subscriber to react to requests and unsubscription.
    2. Next, we retrieve the current known connection and set it on the PublishProducer so it can call the drain() method if it wishes.
    3. We attempt to add the PublishProducer to the internal tracking array. If this fails, it means the current connection has terminated and we have to try the next connection (once becomes available) by looping a bit.
    4. Even if the add succeeded, the child might have just unsubscribed and thus the remove might not have found it. By calling it here again, we can make it sure the PublishProducer doesn't stay in the array unnecessarily.
    5. Once the add succeeded, we have to call drain since a concurrent call in PublishProducer might have not seen a non-null connection and couldn't notify the connection for more values (or about unsubscription). The call will make sure this PublishProducer is handled as necessary.


    State.connect

    This method is responsible for triggering a single connection on an unconnected Connection instance and/or return the Subscription that let's an active Connection get unsubscribed.


    public void connect(Action1<? super Subscription> disconnect) {
    for (;;) {
    Connection<T> curr = this.connection.get();

    if (!curr.connected.get() &&
    curr.connected.compareAndSet(false, true)) { // (1)
    curr.doConnect(disconnect);
    return;
    }
    if (!curr.parent.isUnsubscribed()) { // (2)
    disconnect.call(curr.parent);
    return;
    }

    replaceConnection(curr); // (3)
    }
    }

    This method is also racing with a termination/disconnection and as such, it has to take them into account when attempting to establish a fresh connection.


    1. It works by first retrieving the current connection and if the current thread is the first, switch it into a connected state. If successful, the doConnect method is called which will do the necessary subscription work.
    2. Otherwise, check if the current connection is unsubscribed. If not return it to the callback. Note that there is a small window here where the current connection is determined active but may become disconnected/terminated when the method is called. Resolving this issue requires either blocking synchronization between termination and connection or other serialization approach. In practice, however, this is rarely an issue and can be ignored.
    3. Finally, if the current connection is disconnected, let's replace it with a fresh, not-yet connected Connection and try the loop again.

    Connection.createParent

    The method constructs a SourceSubscriber and sets it up to behave according to the disconnection strategy:

    SourceSubscriber createParent() {
    SourceSubscriber parent = new SourceSubscriber<>(this);

    parent.add(Subscriptions.create(() -> {
    switch (state.strategy) {
    case SEND_COMPLETED:
    onCompleted();
    break;
    case SEND_ERROR:
    onError(new CancellationException("Disconnected"));
    break;
    default:
    disconnected = true;
    drain();
    }
    }));

    return parent;
    }

    The method will instantiate a SourceSubscriber and add a Subscription to it. This subscription, depending on the disconnection strategy, will either call onCompleted, onError with a CancellationException or set the disconnect flag followed by a call to drain (the onXXX methods call drain()).

    We need the disconnected flag because we can't use an isUnsubscribed check: it would always skip the terminal event and appear as if we'd have the NO_EVENT strategy.


    Connection.add, Connection.remove

    The algorithms for adding and removing resources to an array-based container with copy-on-write semantics should be quite familiar by now. For completeness, here are the methods anyway:


    boolean add(PublishProducer<T> producer) {
    for (;;) {
    PublishProducer<T>[] curr = subscribers.get();
    if (curr == TERMINATED) {
    return false;
    }

    int n = curr.length;

    PublishProducer<T>[] next = new PublishProducer[n + 1];
    System.arraycopy(curr, 0, next, 0, n);
    next[n] = producer;
    if (subscribers.compareAndSet(curr, next)) {
    return true;
    }
    }
    }

    void remove(PublishProducer<T> producer) {
    for (;;) {
    PublishProducer<T>[] curr = subscribers.get();
    if (curr == TERMINATED || curr == EMPTY) {
    return;
    }

    int n = curr.length;

    int j = -1;
    for (int i = 0; i < n; i++) {
    if (curr[i] == producer) {
    j = i;
    break;
    }
    }

    if (j < 0) {
    break;
    }
    PublishProducer<T>[] next;
    if (n == 1) {
    next = EMPTY;
    } else {
    next = new PublishProducer[n - 1];
    System.arraycopy(curr, 0, next, 0, j);
    System.arraycopy(curr, j + 1, next, j, n - j - 1);
    }
    if (subscribers.compareAndSet(curr, next)) {
    return;
    }
    }
    }

    Connection.onXXX

    The four onXXX methods on the class are quite sort, therefore, I'll show them togheter in this subsection:


    void onConnect(
    Action1<? super Subscription> disconnect) { // (1)
    disconnect.call(this.parent);

    state.source.unsafeSubscribe(parent);
    }

    @Override
    public void onNext(T t) { // (2)
    if (queue.offer(t)) {
    drain();
    } else {
    onError(new MissingBackpressureException());
    parent.unsubscribe();
    }
    }

    @Override
    public void onError(Throwable e) {
    if (!error.compareAndSet(null, e)) { // (3)
    e.printStackTrace();
    } else {
    done = true;
    drain();
    }
    }

    @Override
    public void onCompleted() { // (4)
    done = true;
    drain();
    }

    Let's see them:


    1. The reason we have to drag the Action1 all the way here instead of calling it State.connect at (2) is that the call must happen before the actual subscription to the underlying Observable to allow synchronous cancellation.
    2. The next method offers the value and calls drain to make sure it is delivered if possible. Note that if the queue is full, we reward it with a MissingBackpressureException and unsubscription; it means the upstream doesn't handle backpressure well or at all.
    3. Since we may receive an error as part of the upstream event as well as a disconnection event, we heed an AtomicReference and set only one of them as the terminal event. In this example, the first one wins, the other gets printed to the console. If the CAS succeded, we set the done flag and call drain to handle things.
    4. It is true onCompleted can also be called from two places, but since it just sets the done flag to true, there is no need for any CAS-ing here. It is also true that due to the disconnection strategy, the onError and onCompleted can race with each other. However, since the difference of handling them is just that error contains null or not, it is't really a problem. Note also that since we used unsafeSubscribe in onConnect, there shouldn't be any call to the SourceSubscriber.unsubscribe coming from upstream and causing trouble if the source terminated normally and the disconnection strategy happen to be SEND_ERROR.

    Connection.drain

    This is unquestionably the heart of the operator and the most complicated logic due to the effects of concurrently changing values it has to rely on. I'll explain it in piece by piece:

    First, it contains a familiar drain loop with wip counter and missed count:

    void drain() {
    if (wip.getAndIncrement() != 0) {
    return;
    }

    int missed = 1;

    for (;;) {

    if (checkTerminated(done, queue.isEmpty())) {
    return;
    }

    // implement rest

    missed = wip.addAndGet(-missed);
    if (missed == 0) {
    break;
    }
    }
    }

    Nothing fancy yet. The wip counter doubles as the serialization entry point on a 0 - 1 transition and a missed counter above that.

    If inside the loop, the first thing to do is to check for a terminal condition via checkTerminated (explained later). It checks for the terminal events and disconnected state and acts accordingly. This is done before the upcoming request coordination since terminal events are not subject to backpressure management and can be emitted before any child Subscriber requests anything.

    The next step is to perform request coordination. Since we set out to do a lockstep coordination, we have to ask all known child subscribers for their current requested amount and figure out the minimum amount everybody can receive. Note that this can be zero.


            //... checkTerminated call

    PublishProducer<T>[] a = subscribers.get();

    int n = a.length;
    long minRequested = Long.MAX_VALUE;

    for (PublishProducer<T> pp : a) {
    if (!pp.isUnsubscribed()) {
    minRequested = Math.min(minRequested, pp.requested.get());
    }
    }

    // ... missed decrementing

    At this point, it is possible n is zero. If there are no subscribers, we set out to "slowly drip away" the available values:


            // ... minRequested calculation

    if (n == 0) {
    if (queue.poll() != null) {
    parent.requestMore(1);
    }
    } else {
    // implement rest
    }

    // ... missed decrementing

    We have to check if the queue is non empty and consume a value with a single poll() then we ask for replenishment. Note that the "slowness" depends on the speed of the upstream Observable. If one decides to do nothing if there are no subscribers, the if statement can be simplified to if (n != 0) { } but should not be removed!

    If we know there are any subscribers and we know the minimum requested amount, we can try draining our queue and emit that amount to everybody.


                // if n != 0 branch

    if (checkTerminated(done, queue.isEmpty())) { // (1)
    return;
    }

    long e = 0L;
    while (minRequested != 0) {

    boolean d = done;
    T v = queue.poll();

    if (checkTerminated(d, v == null)) { // (2)
    return;
    }

    if (v == null) {
    break;
    }

    // final detail to implement

    minRequested--; // (3)
    e++;
    }

    if (e != 0L) { // (4)
    parent.requestMore(e);
    }

    // end of n != branch

    This should also look familiar. We check the  terminal conditions again (1) (optional if you want to be eager). Next, we loop until the minRequested is zero or the queue becomes empty. Inside the loop we do the usual termination checks (2) and emission accounting (3). After the loop, if there were emissions, we ask for replenishment from the SourceSubscriber instance (4).

    Lastly, the final piece of the drain method is the publication of each value to all subscribers:


                    // ... v == null check

    for (PublishProducer<T> pp : a) {
    pp.actual.onNext(v);
    if (pp.requested.get() != Long.MAX_VALUE) {
    pp.requested.decrementAndGet();
    }
    }

    // ... minRequested--

    For each of the PublishProducer (i.e., child Subscriber), we emit the value and decrement the requested amount if not Long.MAX_VALUE (i.e., unbounded child Subscriber).

    Wasn't that painful, was it?


    Connection.checkTerminated

    The checkTerminated method has more things to do since it has to deliver the terminal events to all Subscribers while making sure new Subscribers don't succeed within the add method.


    boolean checkTerminated(boolean done, boolean empty) {    // (1)
    if (disconnected) { // (2)
    subscribers.set(TERMINATED);
    queue.clear();
    return true;
    }
    if (done) {
    Throwable e = error.get();
    if (e != null) {
    state.replaceConnection(this); // (3)
    queue.clear();

    PublishProducer<T>[] a =
    subscribers.getAndSet(TERMINATED); // (4)

    for (PublishProducer<T> pp : a) { // (5)
    if (!pp.isUnsubscribed()) {
    pp.actual.onError(e);
    }
    }


    return true;
    } else
    if (queue.isEmpty()) {
    state.replaceConnection(this); // (6)

    PublishProducer<T>[] a =
    subscribers.getAndSet(TERMINATED);

    for (PublishProducer<T> pp : a) {
    if (!pp.isUnsubscribed()) {
    pp.actual.onCompleted();
    }
    }

    return true;
    }
    }
    return false;
    }

    It works as follows:


    1. The method takes only a done and an empty indicator but not any individual Subscriber or the array of known subscribers.
    2. Since the disconnected flag is set only if the disconnection strategy was NO_EVENT, we can't do much but just set in the TERMINATED indicator array. Anybody unlucky enough still subscribed won't get any further events.
    3. If the done flag is true and there is an error we first replace the current connection with a fresh one (within the state) so newcommers won't try to subscribe to a terminated connection. 
    4. After clearing the queue for any normal values, we swap in the TERMINATED indicator array so ...
    5. ... anybody who got in can now receive its terminal event and the drain loop will quit.
    6. The same logic applies in the case when the upstream has completed normally and the queue has become empty.

    Testing it out

    Finally, we reached the end of one of the most complicated operators in history of RxJava. Now let's reward us via a small unit test to see if the backpressure and the disconnection stategy really works:


    Observable<Integer> source = Observable.range(1, 10);

    TestSubscriber<Integer> ts = TestSubscriber.create(5);

    PublishConnectableObservable<Integer> o = createWith(
    source, DisconnectStrategy.SEND_ERROR);

    o.subscribe(ts);

    Subscription s = o.connect();

    s.unsubscribe();

    System.out.println(ts.getOnNextEvents());
    ts.assertValues(1, 2, 3, 4, 5);
    ts.assertNotCompleted();
    ts.assertError(CancellationException.class);

    It should print [1, 2, 3, 4, 5] to the console and quit without any AssertionErrors. Neat, isn't it?

    Conclusion

    In this lenghtly and brain-stretching blog post, I've explained the requirements and problems around ConnectableObservables that want to do request coordination between its child Subscribers and its upstream Observable. I then showed an implementation of a publish() like ConnectableObservable which features disconnection strategy to avoid hanging its child Subscribers.

    This is, however, not the most complicated operator in RxJava. It isn't replay(), even though the bounded version is a bit more complicated than the PublishConnectableObservable (but only due to the boundary management). It is not the most commonly used operator either and in fact, that is simpler due to fewer state-clashing. No, the most complicated operator to day has so intertwined request coordination that even I'm not sure it is possible to write a buffer-bounded version of it.

    But enough of mysterious foreshadowing! In the next part, I'm going to detail what it takes to implement a replay()-like ConnectableObservable.
    Viewing all 49 articles
    Browse latest View live