Files
gerrit/java/com/google/gerrit/server/quota/DefaultQuotaBackend.java
Patrick Hiesel fc9817f3f3 Add QuotaBackend and QuotaEnforcer extension point
In a recent discussion on the mailing list we agreed that Gerrit-core
would benefit from a mechanism to enforce various kinds of quotas.

As there is no gold-standard for quota checking and there are vastly
different quotas that one might want to enforce (HTTP-QPS,
number-of-projects, numBytesPerProject to name just a few) we provide
this as an extension point for plugins.

This commit implements the extension point in core using AutoValue
requests and responses to keep the interface as stable as possible. We
abstract away from this interface by implementing QuotaBackend and
DefaultQuotaBackend to centralize the plugin calling, provide a fluent
API for Gerrit-core to interact with and be able to provide a different
implementation that doesn't require plugins in tests and other settings
that might have such a need. The returned QuotaResponse.Aggregated
object offers throwing methods to assert that all quota checks passed
and eliminate try-catch-throw boiler-plate in callers.

Decoupling QuotaBackend from QuotaEnforcer also makes it so that we can
improve the fluent interface as we get more callers to make it easy to
use while keeping QuotaEnforcer stable to break as little plugins as
possible.

In the following paragraphs, I'll outline a bit of the thoughts that
went into the design of the interfaces:

For QuotaEnforcer, we don't want to throw exceptions, but have rich
return types instead. One reason is that the general Java world is
moving into the direction of UncheckedExceptions, another reason is that
we can be richer in expressing what happened in a returned object.

QuotaRequestContext contains all state besides the quota group and the
number of tokens we want to deduct. This is because this state might
change and be extended in the future. In this case we want existing
plugins to remain compatible.

Why are we passing Change.Id, Account.Id and Project.NameKey as Java
objects to plugins instead of just encompassing them as IdStrings in
the quota group (/projects/my-project/create)?
Mainly because we don't want plugins to have to parse this back into an
identifier they can use which would be wasteful and add complexity.

Why are we passing these at all?
There are uses cases where it matters which top-level entity the request
is for. For example: Limiting the number of bytes per project. We want
to support these as well.

Why is QuotaEnforcer offering a refill mechanism?
We offer a refill mechanism that basically tries to roll-back the last
quota request. This is only triggered when one of the QuotaEnforcers
denied the request, but other succeed. In this case, the succeeded ones
might have deducted quota for the request, even though others did not
and we let the request fail because of a lack of quota. In this case
we want to try to undo the action. This is best-effort and plugins
can choose not to do refilling.

Why is QuotaBackend *not* offering a refill mechanism?
We don't want to pollute core-Gerrit with try-catch-finally blocks that
refill quota. This is a design decision to keep the code simple. An
action in Gerrit that failed after it passed a quota check will have
deducted quota. Most QuotaEnforcer implementations are refilling, so
this is only a cosmetic issue that vanishes fast. Overall, we see the
QuotaBackend as a monolith, that can support internal rollbacks without
offering this mechanism to external callers.

In subsequent commits, we want to add quota checks to Gerrit-core and
adapt the Quota Plugin to use the new system instead of other hooks as
it does right now.

We will also provide tests where applicable.

Change-Id: I0ac9d4de583871f92040db55f0df29465029ad55
2018-12-04 13:23:18 +01:00

150 lines
5.6 KiB
Java

// Copyright (C) 2018 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.server.quota;
import static com.google.common.base.Preconditions.checkState;
import com.google.common.collect.ImmutableList;
import com.google.common.flogger.FluentLogger;
import com.google.gerrit.reviewdb.client.Account;
import com.google.gerrit.reviewdb.client.Change;
import com.google.gerrit.reviewdb.client.Project.NameKey;
import com.google.gerrit.server.CurrentUser;
import com.google.gerrit.server.plugincontext.PluginSetContext;
import com.google.gerrit.server.plugincontext.PluginSetEntryContext;
import java.util.ArrayList;
import java.util.List;
import javax.inject.Inject;
import javax.inject.Provider;
import javax.inject.Singleton;
@Singleton
public class DefaultQuotaBackend implements QuotaBackend {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
private final Provider<CurrentUser> userProvider;
private final PluginSetContext<QuotaEnforcer> quotaEnforcers;
@Inject
DefaultQuotaBackend(
Provider<CurrentUser> userProvider, PluginSetContext<QuotaEnforcer> quotaEnforcers) {
this.userProvider = userProvider;
this.quotaEnforcers = quotaEnforcers;
}
@Override
public WithUser currentUser() {
return new WithUser(quotaEnforcers, userProvider.get());
}
@Override
public WithUser user(CurrentUser user) {
return new WithUser(quotaEnforcers, user);
}
private static QuotaResponse.Aggregated request(
PluginSetContext<QuotaEnforcer> quotaEnforcers,
String quotaGroup,
QuotaRequestContext requestContext,
long numTokens,
boolean deduct) {
checkState(numTokens > 0, "numTokens must be a positive, non-zero long");
// PluginSets can change their content when plugins (de-)register. Copy the currently registered
// plugins so that we can iterate twice on a stable list.
List<PluginSetEntryContext<QuotaEnforcer>> enforcers = ImmutableList.copyOf(quotaEnforcers);
List<QuotaResponse> responses = new ArrayList<>(enforcers.size());
for (PluginSetEntryContext<QuotaEnforcer> enforcer : enforcers) {
try {
if (deduct) {
responses.add(enforcer.call(p -> p.requestTokens(quotaGroup, requestContext, numTokens)));
} else {
responses.add(enforcer.call(p -> p.dryRun(quotaGroup, requestContext, numTokens)));
}
} catch (RuntimeException e) {
logger.atSevere().withCause(e).log("exception while enforcing quota");
responses.add(QuotaResponse.error(e.getMessage()));
}
}
if (deduct && responses.stream().anyMatch(r -> r.status().isError())) {
// Roll back the quota request for all enforcers that deducted the quota (= the request
// succeeded). Don't touch failed enforcers as the interface contract said that failed
// requests should not be deducted.
for (int i = 0; i < responses.size(); i++) {
if (responses.get(i).status().isOk()) {
enforcers.get(i).run(p -> p.refill(quotaGroup, requestContext, numTokens));
}
}
}
logger.atFine().log(
"Quota request for %s with %s (deduction=%s) for %s token returned %s",
quotaGroup,
requestContext,
deduct ? "(deduction=yes)" : "(deduction=no)",
numTokens,
responses);
return new AutoValue_QuotaResponse_Aggregated(ImmutableList.copyOf(responses));
}
static class WithUser extends WithResource implements QuotaBackend.WithUser {
WithUser(PluginSetContext<QuotaEnforcer> quotaEnforcers, CurrentUser user) {
super(quotaEnforcers, QuotaRequestContext.builder().user(user).build());
}
@Override
public QuotaBackend.WithResource account(Account.Id account) {
QuotaRequestContext ctx = requestContext.toBuilder().account(account).build();
return new WithResource(quotaEnforcers, ctx);
}
@Override
public QuotaBackend.WithResource project(NameKey project) {
QuotaRequestContext ctx = requestContext.toBuilder().project(project).build();
return new WithResource(quotaEnforcers, ctx);
}
@Override
public QuotaBackend.WithResource change(Change.Id change, NameKey project) {
QuotaRequestContext ctx = requestContext.toBuilder().change(change).project(project).build();
return new WithResource(quotaEnforcers, ctx);
}
}
static class WithResource implements QuotaBackend.WithResource {
protected final QuotaRequestContext requestContext;
protected final PluginSetContext<QuotaEnforcer> quotaEnforcers;
private WithResource(
PluginSetContext<QuotaEnforcer> quotaEnforcers, QuotaRequestContext quotaRequestContext) {
this.quotaEnforcers = quotaEnforcers;
this.requestContext = quotaRequestContext;
}
@Override
public QuotaResponse.Aggregated requestTokens(String quotaGroup, long numTokens) {
return DefaultQuotaBackend.request(
quotaEnforcers, quotaGroup, requestContext, numTokens, true);
}
@Override
public QuotaResponse.Aggregated dryRun(String quotaGroup, long numTokens) {
return DefaultQuotaBackend.request(
quotaEnforcers, quotaGroup, requestContext, numTokens, false);
}
}
}