ARTICLE AD BOX
We are trying to build a data transformation engine. Our requirements are
We will have one or more datasource, sql query or excel upload
We will configure set of rules,
update value of some properties in one dataset with constant or from a value from another dataset where condition match
delete rows where certain condition match from a dataset
insert rows in a dataset from another dataset
All these rule will be preconfigured and saved in a database.
Example:
A package with a datasource of sql query. Set of rules include
Replace value of col1 with "XXX"
Replace value of col2 with "yyy" where ColId = 1
At the moment we are doing it by iterating over every row and every column for every rule, which works but as you can image is very slow.
Is it possible to store these rule as some sort of linq expression tree and apply them on the data. Any suggestions or pointers are appreciated.
