Optimizers

.. py:class:: CompositeOptimizer(children: dict[str, ~torch.optim.optimizer.Optimizer])

module:

silverspoon_kd.optim

canonical:

silverspoon_kd.optim.composite_optimizer.CompositeOptimizer

Bases: :py:class:~silverspoon_kd.optim._composite_base._CompositeBase, :py:class:~torch.optim.optimizer.Optimizer

Wraps N per-student optimizers as a single torch.optim.Optimizer.

Inheriting from Optimizer ensures isinstance checks pass and accelerator.prepare() wraps it correctly via AcceleratedOptimizer.

Each child optimizer’s param_groups are tagged with metadata so they can be disaggregated for state_dict / load_state_dict.

param children:

Mapping from student name to its optimizer.

.. py:method:: CompositeOptimizer.init(children: dict[str, ~torch.optim.optimizer.Optimizer])

module:

silverspoon_kd.optim

.. py:method:: CompositeOptimizer.step(closure: ~collections.abc.Callable[[], float] | None = None) -> float | None

module:

silverspoon_kd.optim

Delegate step() to active children only.

.. py:method:: CompositeOptimizer.zero_grad(set_to_none=True)

module:

silverspoon_kd.optim

Delegate zero_grad() to all children (not just active).

.. py:property:: CompositeOptimizer.is_overflow

module:

silverspoon_kd.optim

Return False — no overflow detection in composite optimizer.

.. py:class:: CompositeScheduler(children: dict[str, ~typing.Any])

module:

silverspoon_kd.optim

canonical:

silverspoon_kd.optim.composite_scheduler.CompositeScheduler

Bases: :py:class:~silverspoon_kd.optim._composite_base._CompositeBase

Wraps N per-student LR schedulers as a single scheduler.

Does not need to inherit from a base class — the Trainer only calls .step(), .state_dict(), and .load_state_dict().

param children:

Mapping from student name to its LR scheduler.

.. py:method:: CompositeScheduler.init(children: dict[str, ~typing.Any])

module:

silverspoon_kd.optim

.. py:method:: CompositeScheduler.step(*args, **kwargs)

module:

silverspoon_kd.optim

Delegate step() to active children only.

.. py:method:: CompositeScheduler.get_last_lr() -> list

module:

silverspoon_kd.optim

Concatenate get_last_lr() from active children.