Persistent Hierarchies and Key Generators

We had a very hot discussion with Alex Yakunin about this strange DataObjects.Net behavior. How can you describe his solution of the problem? Is it workaround or clear and solid solution? Why such problem appeared at all?

In this post I'll describe just my own opinion.

First of all, let's remember that DO4 model can be divided into several persistent hierarchies. Any non-abstract persistent type must belong to one of them. In general hierarchies can be completely defined as sets of classes with following peculiarities:

• All types(tables) within a single hierarchy has identical key structure.
• All entities within a hierarchy has unique key value

So, different hierarchies can have different key structures and entity from one hierarchy can have the same key value as entity from another. 'Key' type that uniquely identifies any entity within a domain originally contained two fields: 'Hierarchy' and 'KeyValue'.

If you declare persistent field as a reference to some entity, you must explicitly specify hierarchy you are referencing to. I.e. you can not create field of 'Entity' type, but can create it of 'Person' type, because two different entities with given key value can exist, but two persons not.

OK, we have very solid concepts here, everything is clear, but...

...several months ago, Xtensive headquarter, Ekaterinburg... DO4 team meeting...

- Hi, guys! Our customers ask us when "Persistent Interfaces" (cool feature of DO3) will be implemented in DO4. I think it's time to do this.
- Sure, Persistent Interfaces is such a cool feature! Must have!
- OK, let's say some types from our model can implement some persistent interface, we will be able to query them in unified way and declare persistent fields as references to such interfaces.
- But if we could do such references, we would know which hierarchy we are referencing, i.e. all implementers of persistent interface must belong to a single hierarchy. Is it cool enough?
- Surely no.
- I have the funny idea!!! We have a set of key generators in our database, generally one key generator can generate keys for several hierarchies, in such case all these hierarchies has similar key structures and unique key values. So we can require implementers of any persistent interface to use common key generator, but they still can belong to different hierarchies. That's decided.
- OK, but there is still one little problem. When we are about to resolve reference to persistent interface we must create a 'Key' instance to fetch an entity, but we don't know exact hierarchy.
- Yes, but we surely know key generator, cause all implementers of our interface has common one. Let's replace 'Hierarchy' property in 'Key' type with 'KeyGenerator'.
- OK, I'll do this immediately.

By this time we have very cool feature and two interesting concepts: Persistent Hierarchy and Key Generator. My question is: What's the difference between them.

• Both concepts divide persistent classes into several groups
• In both cases classes within a group has common key structure
• In both cases entities within a group has unique key values

As a fact we use key generators as a persistent hierarchies in most cases, and it leads to misunderstanding.

So in example Alex Yakunin described in his blog we faced the problem generated by that decision to mix hierarchies and key generators. Customer knows that he should declare different hierarchies if he uses identical key values, and he did, but keys are still equal. Why to use hierarchies at all? Are they still make sense or not?

I think, that it will be more effective to require all persistent interface implementers to belong to a single persistent hierarchy and don't mix those two concepts. Surly there are some different solutions, but anyway we should keep our architecture as clear and simple as possible.

New WPF sample

In the end of December 2009 I announced new DataObjects.Net sample of WPF application. Today I've committed its source code in our repository. Application is not ideal yet, there are several known bugs and many things to improve.

I plan to describe how it works in this blog a bit later. You can see what it looks like on following picture and browse its source code in our repository.

WPF applications and DataObjects.Net

During last two weeks I've been working on new WPF sample for DataObjects.Net. In contrast to existing WPF-sample it will embrace wide variety of scenarios usually used in WPF or WinForms applications. The first version of the sample will be built as 2-tier application using DisconnectedState approach. It will be mainly focused on following aspects:

 - Working with Sessions and DisconnectedStates
 - Managing WPF events
 - Data binding

By this moment I am communicating with several community members who use DataObjects.Net in WPF applications. There are a lot of questions and support requests related to these aspects.

By the way, I'd like to provide two versions of this application: Plain WPF and MVVM (Model-View-ViewModel pattern) version. Surely MVVM  is the most attractive pattern to use in WPF application. On other hand, it's rather difficult to find good real-life MVVM implementation example.

My optimistic plan implies that some ready for use sample will be published till the New Year.

"Soft Delete" (aka "Logical Delete") in ORM

People often ask which ORM supports "Soft Delete" or "Logical Delete". Let's try to figure out what is it and whether ORM should internally support it.

"Soft Delete" is a way of removing business entities, which implies that we mark an entity as deleted instead of physical removing it from database. This approach is often used in Line of Business applications because of its several advantages:

• It allows us to keep history for different  auditing sceneries. For example, somebody removed one document in the past from our workflow system, we surely want to be able to audit removing log and data removed document contained.
• It allows to implement Recycle Bin approach in easy way. We'd like to be able to recycle any document removed in the past.

To implement "Soft Delete" feature in a simple case we should:

• Create IsDeleted persistent property of bool type in all softly removable types.
• Automatically filter all queries, i.e. automatically add Where(entity => !entity.IsDeleted) to each LINQ query.
  

In my example on DataObjects.Net I use single base class for all business objects, so I can add IsDeleted field to this class:

public class BusinessObject : Entity
{
  [Field]
  public bool IsDeleted { get; set;}

  public new void Remove()
  {
    IsDeleted = true;
  }
}

Then let's create DataContext class responsible for data access:

public static class DataContext
{
  public static IQueryable<T> GetAll<T>()
    where T : BusinessObject
  {
    return Query<T>.All
      .Where(entity => !entity.IsDeleted);
  }
}

Now we can softly remove our entities and query not removed ones. I've created small sample illustrating model of blog-publishing service, consisting of three classes: Blog, BlogPost and Comment. So I can query recent posts from my blog using such LINQ-query:

from post in DataContext.GetAll<BlogPost>()
where
  post.PublishDate > DateTime.Now-TimeSpan.FromDays(7) &&
  post.Blog == myBlog
select post;

This query will return all posts except softly deleted and I don't have to add appropriate check to every query in my application.

Generally, I am sure that "Soft Delete" shouldn't be internally implemented within an ORM framework, because it's easy to implement it yourself and your own implementation will be more flexible than built-in. Why flexibility is important? In my example I've chosen the simplest way of its implementation. In real-life scenarios there are many aspects connected with soft deletion in specific ways, for example:

• Security: You may want to define who has a permission to access removed entities, etc...
• Entities dependency: You may want to consider some entities as deleted when their dependency parent is deleted, so you need to specify more complex filter on queries.

CodeProject

Auto-transactions API

Which auto-transactions API I'd like to have in ideal ORM?

In a typical business application we have a set of session-bound classes, for example our business classes, services, etc... Each method, property or constructor in these classes can be either transactional or not transactional. We should decide which of them will be transactional and inform ORM about this, using a set of attribute-based rules. I'd like these attributes to allow me to specify following rules:

• All members of this class are not transactional by default
• All public methods of this class and all its inheritors are transactional by default
• This method requires new nested transaction even if some transaction is already open
• This member doesn't require transaction
  
• This method requires transaction with "ReadCommited" isolation level

Moreover I'd like API to be very simple and intuitive. It should allow to define default behavior and manage transactional behavior in flexible way.

When I starting to implement my model and business logic layer, I decide which policy to use. I can make all public members in all classes of my model transactional by marking base class as "Transactional", and then mark some members as not transactional by appropriate attribute. On other hand I can make everything not transactional, except public methods in some service classes, and open transactions manually each time I need them. I can use third approach, for example make methods transactional, but properties not. It depends on my solution architecture, and it's impossible to find universal right way.

So I think we need something like this:

[Transactional] attribute that can be applied to a session-bound class or its member. Attribute should have following options:

• Should this attribute be applied to inheritors or not
  
• Which members it should be applied to (methods, properties, constructors)
• Visibility of members this attribute should be applied to (public, internal, protected)
• Are members this attribute applied to transactional or not. Another alternative is to mark not transactional members by special [NotTransactional] attribute
  
• Required isolation level for transaction
• Whether new transaction is required or any existing one is enough.
  

There is a small example that illustrates how I'd like it to work. In this example I've made all entities not transactional and all public methods of DocumentService - transactional by default.

1. [NotTransactional(InheritBehavior = true)]
2. public class MyEntityBase : Entity
3. {
4. }
5.  
6. public class Document : Entity
7. {
8.   [Transactional]
9.   public void BusinessMethod()
10.   {      
11.   }
12.  
13.   [Transactional(Options = TransactionOptions.New,
14.     IsolationLevel = IsolationLevel.Serializable)]
15.   public void AnotherBusinessMethod()
16.   {      
17.   }
18. }
19.  
20. [Transactional(
21.   MembersType = MemberTypes.Method,
22.   MembersVisibility = Visibility.Public)]
23. public class DocumentService : SessionBound
24. {
25.   public Document FindLastDocument()
26.   {
27.   }
28.  
29.   public void RemoveAllDocuments()
30.   {
31.   }
    
32.  
33.   [NotTransactional]
34.   public override string ToString()
35.   {
36.   }
37. }

As you can see MyEntityBase class marked by [NotTransactional] attribute, but if we assume that all classes are not transactional by default we must not mark it by this attribute.

Xtensive company and DataObjects.Net

First thing I'd like to do now is to introduce myself, especially explain what do I do now. You have definitely read this post title, yes, I work at Xtensive company on DataObjects.Net project.

Xtensive is small, but very ambitious company, there are about 20 developers working on several interesting projects. I can mention DataObjects.Net, MEScontrol, LiveUI and HelpServer. There is no need to introduce each of them in details, you can found detailed information on our nice-looking website www.x-tensive.com. But really important thing here is that at least two projects DataObjects.Net and LiveUI are frameworks designed to make software development easier. That is why I am here, it's cool to develop software, but it's more cooler to develop software for software developers. Though I should say I also like to develop end-user business applications very much, I did it before I came to Xtensive and I hope I'll do it in future.

So I work on DataObjects.Net project now. What is it? You can think of it as of Object Relational Mapper, ORM. Yes, yet another ORM. Yet another ORM with great history and some unique features. It was started in 2003 by Alex Yakunin, first version had a great success, it was realy outstanding product for that time. Now DataObjects.Net team contains of eight skilful and experienced developers, headed by Alex Yakunin. A year ago product was fully reimplemented and now we have almost new DataObjects.Net with such new features as: The best LINQ to SQL translator among existing ORMs, really good performance, great support of "model-first" development practice, including unique "model-first" way of database schema upgrade.

However there are still so many things to do, first of all we need to make all its features as easy to use as possible, we also try to make it more extensible and offer ready to use patterns and practices for frictionless developing different kinds of applications. For example my personal goal for this month is to complete DataObjects.Net manual, that is not completed yet. Yes, we still don't have technical writer position in our team, partly because of product specificity (it's not easy to understand all peculiarities if you are not software developer).

Hello World!

I've finally decided to start my blog on software development and I hope it will be a good place for my thoughts and ides.