🥮 mooncakes.io
    Sign in

    moonpool

    Resource pools for MoonBit: which resource to hand out, when to make another, which to let go, guarded by the breakers, limiters and backoff of moonward. The clock and the randomness are passed in, so every sequence is reproducible and none of it touches I/O.

    pool
    connection-pool
    circuit-breaker
    backoff
    moonbit
    Download zip
    Version
    0.3.0
    License
    Apache-2.0
    Repository
    Last updated
    yesterday
    Downloads
    1K

    #moonpool

    Resource pools for MoonBit: which resource to hand out, when to make another, and which to let go. A ward of moonward's parts can stand in front of the backend: a breaker that stops new connections to a failing backend, a token bucket that spaces them out, backoff after a failed one, and a shedder that stops lending while the process is overloaded.

    let pool : @moonpool.Pool[Int32] = @moonpool.Pool::new(
      limits=@moonpool.Limits::new(size=20),
      ward=@moonpool.Ward::new(
        breaker=@moonward.Breaker::new(),
        backoff=@moonward.backoff,
      ),
    )
    let now = 0L
    match pool.take(now, roll=0.42) {
      Ready(conn) => ignore(pool.give(conn, now)) // use it, then give it back
      Stale(conn) => ignore(conn) // too old to use: close it and take again
      Make => pool.made(1, now) // open one, or call pool.unmade(now) if that fails
      Wait => () // all in use: queue the caller
      Later(wait) => ignore(wait) // making is held back this long
      Shed => () // fail fast: the backend is refusing, or this process is overloaded
    }

    #No I/O, and that is the point

    Nothing here opens, closes, sleeps or reads a clock. The pool says what to do with a resource and the caller does it. Time is whatever the caller's clock reads, in nanoseconds; roll is a draw from [0, 1) for the breaker and for backoff jitter; cpu is the processor load in thousandths, for the shedder. Passing all three in is what makes every sequence reproducible: a test drives a pool through an hour in a few lines.

    #The pieces

    What it answers
    Pool[T]Which resource to hand out, whether to make another, which have sat or lived long enough to let go
    WardWhat stands in front of the backend; every part optional, the preset ward has none
    pickWhich of several pools to one service to take from: two drawn at random, the one whose breaker refuses less, then the one with fewer in use
    Backoff, Bucket, Gate, SlidingRe-exported from moonward, where they now live

    #What a ward does

    PartWhenAnswer
    shedderbefore lending at allShed while the process is hot and has more in flight than it has shown it can finish
    backoffbefore making, after a failed makeLater(span) until the delay has passed; the delay grows with each failure in a row
    bucketbefore makingLater(span) until the bucket has a token, so reconnections do not arrive at once
    breakerbefore makingShed with the probability the breaker gives a failing backend

    made tells the breaker a connection succeeded and clears the backoff; unmade tells it one failed. The shedder treats a lent resource as a call in flight and the time from take to give as its response time, which is what go-zero's adaptive shedder measures.

    #The defaults, and where they come from

    Limits: ten live, ten idle, thirty minutes before an idle one is dropped, no maximum age. Ten is HikariCP's and Go's database/sql default; keeping idle equal to size is database/sql's behaviour, so a pool that grew under load does not immediately throw the connections away. The ward's parts carry moonward's presets, which are go-zero's and gRPC's.

    #A note on the newest-first order

    A pool hands out the most recently returned resource, not the oldest. A warm connection is likelier to still be good, and handing out the newest lets the oldest age past keep and be evicted, which is how a pool shrinks again after a burst.

    #A resource is never lost, and never younger than it is

    take hands an idle resource that is past its age back as Stale rather than skipping it, so the caller closes it whether or not the eviction timer has run. The pool remembers when each lent resource was made, recognising it by identity on the way back, so give does not reset its age and life retires a connection that has been reused all along.

    #Install

    moon add moonbitstack/moonpool

    A ward's parts come from moonward, so a pool with a ward needs moon add moonbitstack/moonward as well.

    #Licence

    Apache-2.0. See LICENSE.

    Backoff

    source

    When to try again, and how long to wait first.

    The preset is [backoff]. Build another with [Backoff::new], or update one in place with { ..backoff, attempts: 5 }.

    Bucket

    source

    A token bucket: rate tokens accrue per second up to burst, and each request takes one.

    It is the rate limiter that allows a burst after a quiet period, which is what a limit on a human-facing API wants — nobody should be refused for making two requests in the same second after an hour of making none.

    Time is nanoseconds from the caller's clock, as everywhere in this package.

    Gate

    source

    A count with a ceiling: the concurrency limit a server puts on itself.

    Unlike a rate, this does not depend on time at all — it is how many things are happening at once. A server past its limit answers rather than queues, which is what keeps a queue from becoming the outage.

    Jitter

    source

    How much randomness goes into a delay.

    A retry with no jitter makes every client that failed together try again together, which is how one outage becomes a second one. The three named strategies are the ones AWS's "Exponential Backoff and Jitter" measures:

    Rigidthe computed delay, unchanged. Reproducible, and it synchronises clients
    Fulluniform over [0, cap]. The one that spreads clients furthest, and what gRPC sleeps
    Equalhalf the cap plus uniform over [0, cap/2]. Keeps a floor under the wait
    Decorrelateduniform over [base, previous × 3]. Grows from where it last landed rather than from the attempt number

    Sliding

    source

    A sliding window counter: at most quota events in any window long stretch.

    Where a bucket smooths, a window counts. A quota of "a thousand a day" is a window, not a rate, and answering it with a bucket would let a client spend the whole day's allowance in the first minute.

    The stamps of the events in the window are kept, so the count is exact rather than the approximation a two-bucket scheme gives.

    Limits

    source
    pub(all) struct Limits {
      size : Int
      idle : Int
      keep : 
    Span
      life : 
    Span
    } derive(Eq, 
    Debug    )

    The limits a pool holds itself to.

    The preset is [limits]. Build another with [Limits::new], or update one in place with { ..limits, size: 32 }.

    Limits::equal

    source
    fn Limits::equal(Limits, Limits) -> Bool

    Limits::new

    source
    fn Limits::new(size? : Int, idle? : Int, keep? : 
    Span    , life? : 
    Span    ) -> Limits

    Limits by name, every knob with the preset's value.

    Limits::not_equal

    source
    fn Limits::not_equal(x : Limits, y : Limits) -> Bool

    Limits::to_repr

    source

    Pool

    source
    type Pool[T]

    The bookkeeping of holding a limited number of things.

    It does not open, close, or wait — a caller with a runtime does all three. What it knows is how many are live, which are idle, which are lent and since when each was made, and which have sat or lived long enough to be let go.

    Time is passed in as whatever the caller's clock reads, in nanoseconds; the value's origin does not matter because only differences are used. That is what makes a test drive a pool through an hour in three lines.

    A lent resource is recognised on its way back by identity, which is what every connection handle has. That is how its birth survives the loan: a pool that stamped a returned resource as new would never retire one by age, however old.

    Idle resources are kept as (item, born, rested), lent ones as (item, born,ticket), and each resource being made as the breaker's promise and the shedder's ticket it will settle.

    Pool::busy

    source
    fn[T] Pool::busy(self : Pool[T]) -> Int

    How many are lent out or being made.

    Pool::close

    source
    fn[T] Pool::close(self : Pool[T]) -> Array[T]

    Everything the pool holds idle, emptied out for the caller to close — what a shutdown does. Lent resources are the borrowers' to return; each comes back through give and can be closed then.

    Pool::drop

    source
    fn[T] Pool::drop(self : Pool[T], item : T) -> Unit

    Say a lent resource is not coming back, because it broke.

    This is what keeps the live count honest: without it a failed connection would hold a slot for ever and the pool would stop making new ones.

    Pool::evict

    source
    fn[T] Pool::evict(self : Pool[T], now : Int64) -> Array[T]

    Every idle resource that has sat longer than keep or lived longer than life, removed from the pool and handed back for the caller to close.

    Called on a timer by whoever owns the runtime. take hands a stale one back too, so a resource is never lost for want of the timer having run.

    Pool::give

    source
    fn[T] Pool::give(self : Pool[T], item : T, now : Int64) -> Bool

    Hand one back, keeping the birth it was lent with.

    false means the pool did not keep it and the caller should close it: there were already enough idle. A resource that is broken goes to [Pool::drop] instead, because a pool that kept it would hand it to the next caller.

    A resource the pool does not recognise is one made after a Make without [Pool::made] being called: it fills the room take reserved and is taken as born now.

    Pool::idle

    source
    fn[T] Pool::idle(self : Pool[T]) -> Int

    How many are sitting idle.

    Pool::live

    source
    fn[T] Pool::live(self : Pool[T]) -> Int

    How many resources exist right now: idle, lent, and being made.

    Pool::made

    source
    fn[T] Pool::made(self : Pool[T], item : T, now : Int64) -> Unit

    Register a resource made after take said Make, as lent and born now.

    Pool::new

    source
    fn[T] Pool::new(limits? : Limits, ward? : Ward) -> Pool[T]

    An empty pool.

    Pool::refusal

    source
    fn[T] Pool::refusal(self : Pool[T], now : Int64) -> Double

    The share of new connections the ward's breaker is refusing at now; zero without a breaker.

    Pool::take

    source
    fn[T] Pool::take(self : Pool[T], now : Int64, roll? : Double, cpu? : Int) -> Taken[T]

    Take one, or be told to make one, or to wait — or be handed one that is too old to use.

    The most recently returned resource is handed out first. That is not arbitrary: a warm connection is likelier to still be good than a cold one, and handing out the newest lets the oldest age past keep and be evicted, which is how a pool shrinks after a burst.

    Stale is an idle resource past its age. It is already out of the pool and out of the count; the caller closes it and takes again. Handing it back rather than dropping it is the point — a pool that discarded it silently would leak whatever it holds whenever the eviction timer had not run first.

    Pool::unmade

    source
    fn[T] Pool::unmade(self : Pool[T], now : Int64, roll? : Double) -> Unit

    Give back a reservation that did not become a resource: the attempt to make it failed. The breaker counts the failure, and with a Backoff in the ward no Make is handed out again until its delay has passed; roll draws the jitter.

    Taken

    source
    pub(all) enum Taken[T] {
      Ready(T)
      Stale(T)
      Make
      Wait
      Later(
    Span    )
      Shed
    }

    What a pool answers when asked for a resource.

    It never blocks and never opens anything: waiting and connecting both need a runtime, and this package has none. Make is the pool saying there is room for another, and reserving it; Wait is it saying there is not, and the caller's queue decides what that means. Stale is an idle resource too old to use, handed back so the caller can close it before asking again.

    The last two come only from a [Ward]: Later is making held back for that long, by the bucket or by backoff after a failure; Shed is the breaker refusing new connections to a failing backend, or the shedder refusing to lend while the process is overloaded.

    Ward

    source

    What stands in front of a pool's backend. Every part is optional and the preset has none, which leaves the pool exactly as it is without a ward.

    Ward::new

    source

    backoff

    source

    The preset: three attempts, a hundred milliseconds doubling to a one-second ceiling, spread by full jitter.

    Those are gRPC's documented defaults for a retry policy — maxAttempts 3, initialBackoff 0.1s, maxBackoff 1s, backoffMultiplier 2 — and gRPC sleeps a value drawn uniformly from [0, cap], which is full jitter.

    forever

    source

    A span meaning "no limit", which is what zero is taken to mean for the two ages: a resource with no maximum age is never too old.

    limits

    source
    let limits : Limits

    The preset: ten live at once, ten of them allowed to sit idle, discarded after thirty minutes idle and never by age.

    Ten is HikariCP's and database/sql's default pool size, and thirty minutes is database/sql's SetConnMaxIdleTime in the shape most deployments set it. Keeping idle equal to size is database/sql's behaviour too, so a pool that grew under load does not immediately throw the connections away.

    pick

    source
    fn[T] pick(pools : ArrayView[Pool[T]], now : Int64, first~ : Double, second~ : Double) -> Int

    Which of several pools to one service to take from: two drawn by first and second, each in [0, 1), and of those the one whose breaker refuses less, then the one with fewer in use.

    Two random choices spread load almost as evenly as scanning every pool, which is why tower's balance and Finagle make this choice. -1 when there are no pools.

    ward

    source
    let ward : Ward

    Source Files

    Powered by MoonBit

    Site sourceReport issuePackagesBuild queueSkillsStatistics

    © 2026 mooncakes.io