Have a look at the results below.

mysql> show create table product_property_test;
+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table                 | Create Table                                                                                                                                                                                                                                                                                                                                                                                                            |
+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| product_property_test | CREATE TABLE `product_property_test` (
  `id` int(10) unsigned NOT NULL AUTO_INCREMENT,
  `product_id` int(10) unsigned NOT NULL,
  `property_id` int(10) unsigned NOT NULL,
  `value` int(11) NOT NULL,
  `text` text,
  PRIMARY KEY (`id`),
  KEY `product_id` (`product_id`,`property_id`),
  KEY `property_id` (`property_id`,`value`)
) ENGINE=InnoDB AUTO_INCREMENT=79187 DEFAULT CHARSET=utf8 ROW_FORMAT=DYNAMIC |
+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

mysql> SELECT COUNT(*) FROM `product_property_test`;
+----------+
| COUNT(*) |
+----------+
|    47340 |
+----------+
1 row in set (0.04 sec)

mysql> SELECT product_id FROM `product_property_test` WHERE ( property_id =65854 AND value =5493 ) OR ( property_id =65854 AND value =5491 ) OR ( property_id =65863 AND value =5498 ) GROUP BY product_id HAVING COUNT(*) = 3;
+------------+
| product_id |
+------------+
|       9541 |
|      46210 |
|      74013 |
|      88603 |
|      88717 |
|     209541 |
|     246210 |
|     274013 |
|     288603 |
|     288717 |
|     409541 |
|     446210 |
|     474013 |
|     488603 |
|     488717 |
|     609541 |
|     646210 |
|     674013 |
|     688603 |
|     688717 |
|     809541 |
|     846210 |
|     874013 |
|     888603 |
|    1046210 |
|    1246210 |
|    1446210 |
|    1646210 |
+------------+
28 rows in set (0.00 sec)

mysql> EXPLAIN SELECT product_id FROM `product_property_test` WHERE ( property_id =65854 AND value =5493 ) OR ( property_id =65854 AND value =5491 ) OR ( property_id =65863 AND value =5498 ) GROUP BY product_id HAVING COUNT(*) = 3;
+----+-------------+-----------------------+-------+---------------+-------------+---------+------+------+----------------------------------------------+
| id | select_type | table                 | type  | possible_keys | key         | key_len | ref  | rows | Extra                                        |
+----+-------------+-----------------------+-------+---------------+-------------+---------+------+------+----------------------------------------------+
|  1 | SIMPLE      | product_property_test | range | property_id   | property_id | 8       | NULL |  124 | Using where; Using temporary; Using filesort |
+----+-------------+-----------------------+-------+---------------+-------------+---------+------+------+----------------------------------------------+
1 row in set (0.00 sec)

So it’s finding 124 instances of one of the property/value combinations. It needs to create a temp table and do a filesort for the group by and having. Now if that value was a lot bigger, like > 10000, things would start to slow down. Also if you would filter on a large number of properties, the temp table will grow.

Remember that there are a lot of different properties, so one of these properties is only used in a small subset of the product. If let’s say > 5% of the products has a specific property, we should just add a column to the product table.

Now in reality you’d probably not just search using the properties, but filter on fields from the product table as well. This should also limit the number of matching property/value pairs found.

SELECT product.* FROM product INNER JOIN product_property_test q ON product.id = q.product_id WHERE product.category_id = 200 AND ( ( property_id =65854 AND value =5493 ) OR ( property_id =65854 AND value =5491 ) OR ( property_id =65863 AND value =5498 ) ) GROUP BY product_id HAVING COUNT(*) = 3

So in practise this will scale well up to a couple million products. If you have more than that you should look at a different solution, like combining Redis and MySQL.

@Jaimie, I hope your questions are hereby answered. I agree that this isn’t the right way to go for all occasions, but it’s definitely useful for the situation as described above. If you want to discuss this further, please come with practical examples of where, why and how problems will arise.

I don’t believe I was flaming (or even worked up), but you’re still pretty far gone and incorrect. You would probably do better by deconstructing those values into integers and throwing them into a fulltext index. Just use boolean mode and do +100 +51 +600, etc.

The only thing that bothered me is that you’re recommending people don’t serialize things in a database like it was the word of God and he thinks EAV is better. It’s really not, and EAV indexes are barely indexes in the traditional sense. You’re wasting your time esp. if you don’t need an index.

You could use Sphinx if you consider that a storage engine, or you could take the FF approach http://backchannel.org/blog/friendfeed-schemaless-mysql

What you’re doing will not scale for faceted search. Stop recommending it to people

First of all, I don’t understand why you’re all worked up and flaming on a year old article, but I’ll respond anyway.

The problem space is the following. Let’s say we’re a hardware warehouse. We have a lot of different products. They have some common properties like a price, but most properties that are specific to that type of hardware. For example a drill has different properties as a door, which is again different from a can of paint.

Let’s say there are 1000 different kind of products, each product has got about 20 properties, which are not all unique. Still there might be something like 10.000 different properties. Creating a table with 10.000 columns is not feasible, not even it they’re virtual.

If we just needed to store and display those properties, just serializing them would be a good solution. But in our case we want to filter on these properties.

If this is the main part of your project and you have the opportunity to move to a document-oriented DB, I would strongly suggest you do so. However, if this is only a small part of your project, moving everything into a doc DB, might just not be worth it. RDBMSs do have some signification advantages.

So, please take this as a given fact. We need to use an RDBMS and we need to store and filter those 10.000 different kind of properties.

A lot of people will just create a table with a varchar property field and a varchar value field. While it might work, that will indeed be very slow. Another common way is to create a table or field per type, but this also has mayor drawbacks as discussed above.

I’m giving a solution where we cast any worth filtering down to an integer. So the idea is basically Java vs C. In Java a variable knows it’s type, while in C a byte is just 8 bits and the meaning of those 8 bits can only be seen in context of the code. The same thing here, whether the integer represents a date, a decimal or a number, can only be seen in context of the code.

So I agree that we’re abusing instead of using MySQL here. This is a very usable and relatively fast solution though. Much more that the other RDBMS alternatives.

I hope this answers your question and leaves you satisfied

I don’t get your logic. Don’t use serialized fields because they’re an anti-pattern, but EAV is OK because it’s quasi-normalized … but STILL an anti-pattern.

That logic is flawed. If you don’t need an index, and you’re not terribly interested in atomicity and/or can update the data in place using functions within a lock, it’s just dandy. If you embed JSON manipulation functions inside a RDBMS you’re fine. Same for CSV. I do it all the time. It’s fine.

In fact, MariaDB has VIRTUAL COLUMNS that are based in a serialized BLOB approach with options to pull it out to a column if you want it indexed. If not, it’s left as a blob.

Why does the concept of normalization/atomicity change when you go to MongoDB. MongoDB just supports only one table and uses an imperative language instead of a declarative one. Nothing changes. Using MongoDB doesn’t suddenly make not making values atomic correct. The question is whether you care. There are no Gods who will smite you. Just make sure you know what you’re doing.

If you think otherwise, you’re just plain wrong.

EAV is an awful idea invented by people that have no idea what Codd meant. It’s stupid slow and won’t scale.

Even if you have a lot of data and little memory, loading everything isn’t necessarily a bad thing. Your server will simply store the least used data in swap space. Which is okay if that data is hardly used.

Make sure you read the FAQ. It has a lot of info about this: http://code.google.com/p/redis/wiki/FAQ

I don’t really like CouchDB. It’s to abstract, so you end up writing a lot of code just getting the data from the DB. MongoDB is good for logging type of databases. In general I like Redis best for general purpose solutions http://code.google.com/p/redis/. The project doesn’t have as nice a site as Mongo or Couch, but the software is great .

Most data is store in a usual relational scheme, EAV is going to be used only for custom defined attributes of the subscribers.

Some suguest using fixed table pattern.
http://blog.springsource.com/arjen/archives/2008/01/24/storing-custom-fields-in-the-database/
It looks like we’ll need up to 30 custom fields attributes and using fixed table pattern will require to many indexes per table I suppose.