Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
2ffcd6b
Split all FORM components into reading and writing components.
aolivier23 Feb 27, 2026
0be8f74
Removing features from old prototype branch.
aolivier23 Feb 27, 2026
75fdbc5
Removed old unified Persistence class that is no longer needed. It has
aolivier23 Mar 2, 2026
471b535
Removed old unified Storage class that is no longer used.
aolivier23 Mar 2, 2026
b7ddde1
Renamed storage_util.hpp to storage_utils.hpp to match naming convention
aolivier23 Mar 2, 2026
c0071a7
Merge branch 'main' into form_read_write_split
aolivier23 Mar 2, 2026
e464bda
Got latest FORM Storage tests working with the read/write split.
aolivier23 Mar 2, 2026
3e7360e
Merge branch 'main' into form_read_write_split
aolivier23 Mar 18, 2026
aac5c6a
Merge branch 'main' into form_read_write_split
aolivier23 Mar 20, 2026
23e120b
Apply YAML formatter fixes
github-actions[bot] Mar 23, 2026
0e84f5f
Apply cmake-format fixes
github-actions[bot] Mar 23, 2026
d586ade
Apply clang-format fixes
github-actions[bot] Mar 23, 2026
ee7bae1
Apply ruff fixes
github-actions[bot] Mar 23, 2026
a28c500
Removed accidental RNTuple inclusion from factories. The classes it
aolivier23 Mar 23, 2026
0c07e0a
Merge branch 'main' into form_read_write_split
aolivier23 Mar 23, 2026
4346c13
Merge branch 'form_read_write_split' of https://github.com/aolivier23…
aolivier23 Mar 23, 2026
7c9742a
Removed accidental mention of RNTuple components that was setting off
aolivier23 Mar 23, 2026
82aa92f
Removed attribute for TBranch read container that is not relevant to
aolivier23 Mar 23, 2026
0153e49
Removed Write command in TTree read container that's no longer needed.
aolivier23 Mar 23, 2026
c47a8c6
Apply clang-format fixes
github-actions[bot] Mar 23, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion form/form/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -3,5 +3,5 @@
# External dependencies: find_package( PHLEX )

# Component(s) in the package:
add_library(form form.cpp config.cpp)
add_library(form form_reader.cpp form_writer.cpp config.cpp)
target_link_libraries(form persistence)
47 changes: 0 additions & 47 deletions form/form/form.hpp

This file was deleted.

37 changes: 37 additions & 0 deletions form/form/form_reader.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
// Copyright (C) 2025 ...

#include "form_reader.hpp"

#include <stdexcept>
#include <typeinfo>

namespace form::experimental {

form_reader_interface::form_reader_interface(config::output_item_config const& output_config,
config::tech_setting_config const& tech_config) :
m_pers(nullptr)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggest rename

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about output_item_config -> io_item_config? container_config? I think we're going to need it in both the reader and the writer.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The location information on data to read should not depend on configuration.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other than the (global) input file-names, a reading job does not need to tell FORM where and what objects are stored, FORM should find them. Whereas on output, configuration does determine which outputs are written where.

{
for (auto const& item : output_config.getItems()) {
m_product_to_config.emplace(item.product_name,
form::experimental::config::PersistenceItem(
item.product_name, item.file_name, item.technology));
}

m_pers = form::detail::experimental::createPersistenceReader();
m_pers->configureOutputItems(output_config);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For Reader?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you're correct about this. Will review and probably change. Thanks.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this needs a rename but can't be eliminated without rewriting more of PersistenceReader. We use m_output_items to figure out what file and technology to use when making a Token to get row ID.

m_pers->configureTechSettings(tech_config);
}

void form_reader_interface::read(std::string const& creator,
std::string const& segment_id,
product_with_name& pb)
{

auto it = m_product_to_config.find(pb.label);
if (it == m_product_to_config.end()) {
throw std::runtime_error("No configuration found for product: " + pb.label);

Check warning on line 32 in form/form/form_reader.cpp

View check run for this annotation

Codecov / codecov/patch

form/form/form_reader.cpp#L32

Added line #L32 was not covered by tests
}

m_pers->read(creator, pb.label, segment_id, &pb.data, *pb.type);
}
}
34 changes: 34 additions & 0 deletions form/form/form_reader.hpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
// Copyright (C) 2025 ...

#ifndef FORM_FORM_FORM_READER_HPP
#define FORM_FORM_FORM_READER_HPP

#include "form/config.hpp"
#include "form/product_with_name.hpp"
#include "persistence/ipersistence_reader.hpp"

#include <map>
#include <memory>
#include <string>
#include <typeinfo>
#include <vector>

namespace form::experimental {

class form_reader_interface {
public:
form_reader_interface(config::output_item_config const& output_config,
config::tech_setting_config const& tech_config);
~form_reader_interface() = default;

void read(std::string const& creator,
std::string const& segment_id,
product_with_name& product);

private:
std::unique_ptr<form::detail::experimental::IPersistenceReader> m_pers;
std::map<std::string, form::experimental::config::PersistenceItem> m_product_to_config;
};
}

#endif // FORM_FORM_FORM_READER_HPP
32 changes: 10 additions & 22 deletions form/form/form.cpp → form/form/form_writer.cpp
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
// Copyright (C) 2025 ...

#include "form.hpp"
#include "form_writer.hpp"

#include <stdexcept>
#include <typeinfo>

namespace form::experimental {

form_interface::form_interface(config::output_item_config const& output_config,
config::tech_setting_config const& tech_config) :
form_writer_interface::form_writer_interface(config::output_item_config const& output_config,
config::tech_setting_config const& tech_config) :
m_pers(nullptr)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggest rename

{
for (auto const& item : output_config.getItems()) {
Expand All @@ -17,14 +17,14 @@ namespace form::experimental {
item.product_name, item.file_name, item.technology));
}

m_pers = form::detail::experimental::createPersistence();
m_pers = form::detail::experimental::createPersistenceWriter();
m_pers->configureOutputItems(output_config);
m_pers->configureTechSettings(tech_config);
}

void form_interface::write(std::string const& creator,
std::string const& segment_id,
product_with_name const& pb)
void form_writer_interface::write(std::string const& creator,
std::string const& segment_id,
product_with_name const& pb)
{

auto it = m_product_to_config.find(pb.label);
Expand All @@ -40,9 +40,9 @@ namespace form::experimental {
m_pers->commitOutput(creator, segment_id);
}

void form_interface::write(std::string const& creator,
std::string const& segment_id,
std::vector<product_with_name> const& products)
void form_writer_interface::write(std::string const& creator,
std::string const& segment_id,
std::vector<product_with_name> const& products)
{

if (products.empty())
Expand All @@ -69,16 +69,4 @@ namespace form::experimental {
m_pers->commitOutput(creator, segment_id);
}

void form_interface::read(std::string const& creator,
std::string const& segment_id,
product_with_name& pb)
{

auto it = m_product_to_config.find(pb.label);
if (it == m_product_to_config.end()) {
throw std::runtime_error("No configuration found for product: " + pb.label);
}

m_pers->read(creator, pb.label, segment_id, &pb.data, *pb.type);
}
}
37 changes: 37 additions & 0 deletions form/form/form_writer.hpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
// Copyright (C) 2025 ...

#ifndef FORM_FORM_FORM_WRITER_HPP
#define FORM_FORM_FORM_WRITER_HPP

#include "form/config.hpp"
#include "form/product_with_name.hpp"
#include "persistence/ipersistence_writer.hpp"

#include <map>
#include <memory>
#include <string>
#include <vector>

namespace form::experimental {

class form_writer_interface {
public:
form_writer_interface(config::output_item_config const& output_config,
config::tech_setting_config const& tech_config);
~form_writer_interface() = default;

void write(std::string const& creator,
std::string const& segment_id,
product_with_name const& product);

void write(std::string const& creator,
std::string const& segment_id,
std::vector<product_with_name> const& products);

private:
std::unique_ptr<form::detail::experimental::IPersistenceWriter> m_pers;
std::map<std::string, form::experimental::config::PersistenceItem> m_product_to_config;
};
}

#endif // FORM_FORM_FORM_WRITER_HPP
15 changes: 15 additions & 0 deletions form/form/product_with_name.hpp
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did this change?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will investigate. I agree that it shouldn't change. Thank you for catching this.

Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
#ifndef FORM_FORM_PRODUCT_WITH_NAME_HPP
#define FORM_FORM_PRODUCT_WITH_NAME_HPP

#include <string>
#include <typeinfo>

namespace form::experimental {
struct product_with_name {
std::string label;
void const* data;
std::type_info const* type;
};
}

#endif // FORM_FORM_PRODUCT_WITH_NAME_HPP
7 changes: 4 additions & 3 deletions form/form_module.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
// FORM headers - these need to be available via CMake configuration
// need to set up the build system to find these headers
#include "form/config.hpp"
#include "form/form.hpp"
#include "form/form_writer.hpp"
#include "form/technology.hpp"

#include <cassert>
Expand Down Expand Up @@ -43,7 +43,8 @@ namespace {
}

// Initialize FORM interface
m_form_interface = std::make_unique<form::experimental::form_interface>(output_cfg, tech_cfg);
m_form_interface =
std::make_unique<form::experimental::form_writer_interface>(output_cfg, tech_cfg);
}

// This method is called by Phlex - signature must be: void(product_store const&)
Expand Down Expand Up @@ -102,7 +103,7 @@ namespace {
private:
std::string const m_output_file;
int const m_technology;
std::unique_ptr<form::experimental::form_interface> m_form_interface;
std::unique_ptr<form::experimental::form_writer_interface> m_form_interface;
};

}
Expand Down
2 changes: 1 addition & 1 deletion form/persistence/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Copyright (C) 2025 ...

# Component(s) in the package:
add_library(persistence persistence.cpp)
add_library(persistence persistence_reader.cpp persistence_writer.cpp persistence_utils.cpp)
target_link_libraries(persistence core storage)
40 changes: 40 additions & 0 deletions form/persistence/ipersistence_reader.hpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
// Copyright (C) 2025 ...

#ifndef FORM_PERSISTENCE_IPERSISTENCE_READER_HPP
#define FORM_PERSISTENCE_IPERSISTENCE_READER_HPP

#include <map>
#include <memory>
#include <string>
#include <typeinfo>

namespace form::experimental::config {
class output_item_config;
struct tech_setting_config;
}

namespace form::detail::experimental {

class IPersistenceReader {
public:
IPersistenceReader() {};
virtual ~IPersistenceReader() = default;

virtual void configureTechSettings(
form::experimental::config::tech_setting_config const& tech_config_settings) = 0;

virtual void configureOutputItems(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, I expect that I can remove everything dealing with output config from the reader-flavor components. I'll report back if it doesn't work for some reason.

form::experimental::config::output_item_config const& outputItems) = 0;

virtual void read(std::string const& creator,
std::string const& label,
std::string const& id,
void const** data,
std::type_info const& type) = 0;
};

std::unique_ptr<IPersistenceReader> createPersistenceReader();

} // namespace form::detail::experimental

#endif // FORM_PERSISTENCE_IPERSISTENCE_READER_HPP
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
// Copyright (C) 2025 ...

#ifndef FORM_PERSISTENCE_IPERSISTENCE_HPP
#define FORM_PERSISTENCE_IPERSISTENCE_HPP
#ifndef FORM_PERSISTENCE_IPERSISTENCE_WRITER_HPP
#define FORM_PERSISTENCE_IPERSISTENCE_WRITER_HPP

#include <map>
#include <memory>
Expand All @@ -15,10 +15,10 @@ namespace form::experimental::config {

namespace form::detail::experimental {

class IPersistence {
class IPersistenceWriter {
public:
IPersistence() {};
virtual ~IPersistence() = default;
IPersistenceWriter() {};
virtual ~IPersistenceWriter() = default;

virtual void configureTechSettings(
form::experimental::config::tech_setting_config const& tech_config_settings) = 0;
Expand All @@ -33,16 +33,10 @@ namespace form::detail::experimental {
void const* data,
std::type_info const& type) = 0;
virtual void commitOutput(std::string const& creator, std::string const& id) = 0;

virtual void read(std::string const& creator,
std::string const& label,
std::string const& id,
void const** data,
std::type_info const& type) = 0;
};

std::unique_ptr<IPersistence> createPersistence();
std::unique_ptr<IPersistenceWriter> createPersistenceWriter();

} // namespace form::detail::experimental

#endif // FORM_PERSISTENCE_IPERSISTENCE_HPP
#endif // FORM_PERSISTENCE_IPERSISTENCE_WRITER_HPP
Loading
Loading