Nested Configurations
Hypster enables hierarchical configuration management through the hp.nest()
method, allowing you to compose complex configurations from smaller, reusable components.
For an in depth tutorial, please check out the article on Medium: Implementing Modular-RAG using Haystack and Hypster
nest
Function Signature
nest
Function Signaturedef nest(
config_func: Union[str, Path, "Hypster"],
*,
name: Optional[str] = None,
final_vars: List[str] = [],
exclude_vars: List[str] = [],
values: Dict[str, Any] = {}
) -> Dict[str, Any]
Parameters
config_func
: Either a path to a saved configuration or a Hypster config objectname
: Optional name for the nested configuration (used in dot notation)final_vars
: List of variables that cannot be modified by parent configsexclude_vars
: List of variables to exclude from the configurationvalues
: Dictionary of values to override in the nested configuration
Steps for nesting
Define a parent config and use hp.nest
hp.nest
@config
def qa_config(hp: HP):
# Load and nest LLM configuration
llm = hp.nest("configs/llm.py")
# Add QA-specific parameters
max_context_length = hp.int(1000, min=100, max=2000)
# Combine LLM and QA parameters
qa_pipeline = QAPipeline(
model=llm["model"],
temperature=llm["temperature"],
max_context_length=max_context_length
)
Configuration Sources
hp.nest()
accepts two types of sources:
Path to Configuration File
llm = hp.nest("configs/llm.py")
Direct Configuration Object
from hypster import load
# Load the configuration
llm_config = load("configs/llm.py")
# Use the loaded config
qa_config = hp.nest(llm_config)
Value Assignment
Values for nested configurations can be set using either dot notation or nested dictionaries:
# Using dot notation
qa_config(values={
"llm.model": "sonnet",
"llm.temperature": 0.5,
"max_context_length": 1500
})
# Using nested dictionary
qa_config(values={
"llm": {
"model": "sonnet",
"temperature": 0.5
},
"max_context_length": 1500
})
Hierarchical Nesting
Configurations can be nested multiple times to create modular, reusable components:
@config
def indexing_config(hp: HP):
# Reuse LLM config for document processing
llm = hp.nest("configs/llm.py")
# Indexing-specific parameters
embedding_dim = hp.int(512, min=128, max=1024)
# Process documents with LLM
enriched_docs = process_documents(
llm=llm["model"],
temperature=llm["temperature"],
embedding_dim=embedding_dim
)
@config
def rag_config(hp: HP):
# Reuse indexing config (which includes LLM config)
indexing = hp.nest("configs/indexing.py")
# Add retrieval configuration
retrieval = hp.nest("configs/retrieval.py")
Passing Values to Nested Configs
Use the values
parameter to pass dependent values to nested configuration values:
retrieval = hp.nest(
"configs/retrieval.py",
values={
"embedding_dim": indexing["embedding_dim"],
"top_k": 5
}
)
final_vars
and exclude_vars
are also supported.
Best Practices
Modular Design
Create small, focused configurations for specific components
Combine configurations only when there are clear dependencies
Keep configurations reusable across different use cases
Clear Naming
# Use descriptive names for nestd configs llm = hp.nest("configs/llm.py", name="llm") indexer = hp.nest("configs/indexer.py", name="indexer")
Value Dependencies
# Explicitly pass dependent values retriever = hp.nest( "configs/retriever.py", values={"embedding_dim": embedder["embedding_dim"]} )
File Organization
# Keep related configs in a dedicated directory configs/ โโโ llm.py โโโ indexing.py โโโ retrieval.py โโโ rag.py
Last updated
Was this helpful?