The Analytics Ecology

Back into the fray

Proving that Serendipity is doing it’s job, I’ve had in my mind that it’s time to return to these thoughts and several people contacted me to find out if I was going to return to this blog.

Okay. Into the deep end first.

My time away has been due to busyness. Perhaps some readers have heard, NextStage Received its first patent on its Evolution Technology. For years we’ve been intentionally below the radar, now we seem to be becoming a recognizable object rapidly approaching from the far horizon. Now that we’ve left nap-of-the-earth flying I’m able to discuss things more openly, me thinks, hence some of my responses now and in the future.

Are the visitors happy?

One of the things I did while I was away was talk with a few people (about 100 so far) about what I’ll call The Purpose of Web Analytics. I did this research because of something I wrote in this thread above, “…all these analytics are worthless unless they create happy, satisfied visitors, yes?”

I’ve talked with upper management in education, politics, at national telecoms, financial institutions, transportation, recreation, … a pretty diverse group. Most of them were involved in marketing products or services or some other form of gaining marketshare. None of them were web analysts or involved in web analytics except that they received reports and were expected to act upon them. None of them were particularly happy about being made accountable to a system that (they believed) wasn’t measuring … and here’s where the challenges really made themselves known.

What was being measured? Lots of money was being spent and lots of people were being told that the measurements mattered and as one fellow explained, for the amount of money they were spending they expected some consistency.

“What do you mean by consistency?” I asked.

He pretty much didn’t know. He and those with him said lots of things and it could be distilled to a general dissatisfaction that there wasn’t a single model that they could consistently use and derive actionable meaning from. The dissatisfaction grew geometrically when the discussion got into executives making decisions based on sales presentations rather than a given product’s specific informational abilities.

At one point I leaned towards a speaker and quietly said, “Remember, Joseph friend,” and everybody laughed because the tension in the room was broken.

I reference these anecdotes because one of my original hopes for this platform was an increase in understanding and acceptance of some mutual goals regardless of discipline or tool platform.

In the end, doesn’t it all come down to “…all these analytics are worthless unless they create happy, satisfied visitors…?”

If I can’t act on it, it doesn’t exist

The next item I wish to thread into this discussion comes from an online conversation I had with Critical Mass’s Christopher Berry about why web analytics seems to be a harder sell in Canada than in the US. You can follow my side of the conversation in Canadian Based Business Differences — Responding to June Li, Christopher Berry and Jacques Warren, Responding to Christopher Berry’s Vexing Problem, Part 3 post, The Language of Web Analytics – The Hard(er) Sell in Canada, Responding to Christopher Berry’s “A Vexing Problem, Part 4″ Post, Part 1, Responding to Christopher Berry’s “A Vexing Problem, Part 4″ Post, Part 2 and Communicating Science to Business and Vice Versa and links are provided to Christopher Berry’s side on the conversation in those posts. I’ll invite people to pay particular attention to Communicating Science to Business and Vice Versa because (and as Mr. Berry noted) the summation is what counts, “Business is different. Business (me thinks) tends to be more ‘Tell me how to use this’ hence most business proposals and reports start with Christopher Berry’s nuggets then go into explanations.”

My research is convincing me that (what I recognize as traditional) web analytics is going to be losing its authoritative power in the coming years. I think web analytics (and yes, this does go back to my original hopes for this blog) will evolve (just as anything will if it is going to survive in a given changing environment). What will it do and look like? I have some ideas, of course. Just ideas at present, though. More things to research before putting down on paper (or in a blog) at present.

This does tie into my comment re Avinash Kaushik’s “…we shouldn’t use ill defined engagement metrics as a proxy for something solid like a sale.” I’ve been an oft-times unwilling father-confessor to businesses frustrated by ill-defined metrics of any kind and wanting something that is justifiable a) financially, b) scientifically, c) arithmetically (forget mathematically) and d) produces some kind of “do A, get B”, “this-equals-that” link between action and outcome.

The comment I love about this is “If I can’t act on it then it doesn’t exist”, ie, it’s noise, a distraction at best and something best ignored. This was a wonderful statement used in a business practices discussion.

I’d really enjoy being involved in a web understandability/measurement/future usability discussion that has as its theme “If I can’t act on it then it doesn’t exist.”

“To measure and analyze on and offline behavior and then try to predict who to market to by figuring out what they think is not doable with one tool or one metric.”

I responded earlier to this comment. People who attended either the Toronto ‘08 or SF ‘08 eMetrics conferences are probably well aware by now that NextStage has patented a technology that can determine how someone is thinking through any programmable device. I won’t go deeper into the topic here except to offer a comment I posted on Jim Novo’s blog about the {C,B/e,M} matrix and its use in marketing and analytics.

Picking up where I left off with Jim Novo’s comments in this thread…

I finally had an opportunity to read Jim Novo’s Measuring Engagement and its related Framework for Engagement posts. I truly enjoy Jim’s writing style and the points he makes.

I especially enjoy and appreciate his referencing Relationship Marketing because it places people center stage. Understand people and you can both understand and predict what they’ll do. Watch only what people have done and you can only understand their actions in a specific historical context, you can only predict what they’ll do when the confluence of events that led to their original actions repeats itself. Exactly (and don’t hold your breath). Relationship marketing works at the question “…all these analytics are worthless unless they create happy, satisfied visitors, yes?”

Jim writes “The challenge with this model – and probably why it isn’t more widely known – has been the data, it’s a very analysis-intensive model…”. Yes. Agreed. If Jim (or others familiar with these concepts) is reading (or perhaps at the next conference we meet at), I think this is where being able to substitute cognitive heuristic models makes sense (see Liberation and Heuristics or Responding to Christopher Berry’s “A Vexing Problem, Part 4″ Post, Part 1. I’ve also written elsewhere that I often wonder why more businesses don’t make use of cognitive heuristic models).

For example, I’ve recently been applying heuristic models to helping adult second language learners increase their language acquisition abilities. That’s a traditionally very tough nut to crack and (so far, anyway) I’ve been able to isolate neural activity that tends to make adult language acquisition challenging. Example 2, using heuristic models in the above grew out of learning which heuristic models are used (non-consciously, of course) by which personality types in their decision making processes. This non-conscious heuristic model selection process is being integrated into NextStage’s Rich Personae. These and some other areas of my studies are intensely data-rich models that can be reasonably simplified via cognitive heuristics.

I also strongly like your concept of dis-engagement, although I tend to use a methodology that incorporates “satisfaction” into the scaling system (see Meet Online Engagement’s Little Friend, Satisfaction. I shared that the complete form of this during a discussion at the SF ‘08 eMetrics. It looks something like the figure on the right.

Some definitions to help in understanding; the x-axis is Engagement and is a measure of the amount of pleasure or pain an activity is giving you. If something is giving you either pleasure or pain to any degree your attention is focused on it, hence you are engaged by it according to the definitions documented in Attention, Engagement and Trust: The Internet Trinity and Websites. The y-axis is Satisfaction and is a measure of acceptance and rejection of some internal state and/or external event.

I believe what you are referencing as “dis-engagement” is what we recognize as the slide from high acceptance to “0″ acceptance. Note that this is not rejection (as rejection is an active negation of acceptance) it is a lack of acceptance. I appreciate that the difference might be subtle and I believe that difference is significant. Rejection — the active negation of acceptance — can be thought of as someone pushing something away. Zero-acceptance is the point where one can “take it or leave it” and the internal state and/or external event does not have any value assigned to it, hence doesn’t register strongly in the mind/brain.

Mapping this figure to real world experience, you always want visitors/consumer/etc to be in the first quadrant (where the green curve is). People are both positively engaged (they like what’s going on) and positively satisfied (they accept it gladly). Depending on what you’re selling you may or may not want people in those other quadrants. The second quadrant (bottom yellow curve) indicates someone focusing on painful experiences or information, the fourth quadrant (top yellow curve) indicates someone who finds pleasure in painful experiences or information. The third quadrant (red curve) is where visitors/consumers/etc often end up and marketers/businesses don’t want them to be — the former are actively psychologically and physically moving themselves away from a business/product/service.

I’ll offer that the above is also a reasonable representation of your:
1. Define / Measure Engagement – any way you want to, as appropriate for your business; whatever activity or combinations of activity you feel appropriate
2. Measure dis-Engagement – the absence of Engagement, as in the visitor / customer stopped doing whatever it is you define as Engagement for your business model

I think where the image above (and the math behind it) adds real value is with your “3. Take some kind of Marketing or Service action to slow or reverse the dis-Engagement with dis-Engaging folks” because it provides enough information to know how, exactly, visitors/etc are “right now” interacting with your marketing information.

I also agree whole-heartedly with your statements about predicting “dis-engagement”, etc.. I would love to see the data you used in your example and apply it to the above. I’m willing to bet that satisfaction/acceptance was the real driver (and I won’t get into the depths of group satisfaction/acceptance states here (really, Joseph? You’re going to leave something out? Whatever for?)). I did get a kick out of your graph of email response rates falling over time. It was very similar to the results we found in our research on how to design an effective email newsletter. Bravo! I always love it when our findings match others’. Gives me hope we’re doing something right.

<ASIDE>

For what it’s worth, much of the rest of what you’ve written in your post is so close to what we learned in our email newsletter research that the overlap is astounding. Not surprising, I guess, as you’re listing an email-based experiment. It would be interesting to learn what else the rules we discovered pertain to. Let me know if would like to explore this.
</ASIDE>

You also list an implication about sending different messages to different segments. Yes, agreed. I believe the above allows for much more targeted and action-driven messaging (based on much of what I’ve shared above).

Perhaps, in the end, we’ve derived nothing more than a simplified mathematical model (complete with suggestions for better outcomes) of Relationship Marketing?

Whoosh!

Took me two days to put the above together folks. Sorry for the delay. More to follow. Soon.

Promise.

(still responding to Jim Novo’s 31 Jan 08 10:59am entry…I went off and studied up on things that appeared relevant to the REAN model he was referencing)

I did not know (my apologies, all) that web analytics made use of geometric models (something REAN seems to be). I won’t discuss how the R, E, A and N dimension metrics are collected. Each dimension seems to be some kind of reduction of several other dimensions, though. I’d love to know if there’s some documented formulae for these. My concern is that the problem will be the same western medicine faces when working to understand jun-chen-zuo-shi.

Don’t know if I asked this here or not before…are folks measuring what matters or are they measuring what they can measure and saying it matters? Maybe I need to change the focus of my questions from “what is it called?” to “what is being measured?” This was briefly addressed in the exchange on proxies for the truth. It’s kind of like claiming something is a best practice because it’s been around for a while. Best practice in 1400AD was to sail close to the shore because the edge of the world lay in the other direction.

I’ve been trained to learn an application, distill its principles to extract the theory that makes the application work, then use the theory to develop multiple applications.

That offered, let me restate the question from “What can we measure?” to “What do we want to achieve?”

For example, not “We want to measure engagement” to “We want people to come back to our site.”

Yes, this is a move back to First Principles. One of my mentors, a brilliant mathematician, taught me to always make my mistakes at the beginning of a solution. Errors at the end were usually too difficult to find because you were caught up in the logic inertia. At the beginning of a solution you hadn’t really developed enough of an idea of what the solution looks like to be caught up in the need to see it to its end.

So, with your indulgence and a desire to better understand the task at hand, What do we want to achieve?

(I hope people don’t think I’m out where the buses don’t go when they read my comments. I assure folks, my interests are the same as others — creating reliable metrics. I also hope that my coming from such a different paradigm and with such different trainings will be recognized as opportunities rather than something to be quickly dismissed. One thing I really like about the REAN model is its recognition (as I read it, anyway) that different elements are influencing different things being measured and creating a “collective” measurement. My experience is that measuring isolated processes causes the measurers to ignore the diversity of elements required to understand the situation being observed)

more to follow…

(still responding to Jim Novo’s 31 Jan 08 10:59am entry…I think that’s the one, anyway)

I love your comment “…the need to fit this all into published frameworks because it leads to continuity between disciplines.” Yes. Perhaps I’m seeking the GUT (Grand Unified Theory) of Analytics. Wouldn’t it be grand (no pun intended) though, to find some underlying principles that guide, shape and form these many disciplines?

Or perhaps I’m attempting to create a Double Benioff Zone to understand the web…

Someday at some conference we’ll all meet in the bar and, if you’d like, I’ll bore you to death with an explanation of trans-temporal reafference, how and why it’s probably relevant to any rich content (Flash, audio, video, …). I was going to explain things at a high level here and the high level explanation got to be several pages long, so to heck with that.

I’m pretty sure that trans-temporal reafference is meaningful in a discussion of web pages that are intended to keep a visitor engaged (as I define it) for some period of time longer than a “typical” web page is intended to keep a visitor engaged. Meaning if a web page delivers some content such that the visitor doesn’t need to refresh or move to some other page for a recognizably longer period of time than they spend on most web pages (let’s say they watch a video download that takes 30m of time and a typical page takes 3 seconds to scan and act upon, thus y >> x | x ≈ 0 for calculation purposes).

Anyway…

I’ll accept your statement “You have various levels of Engagement, a continuum from Highly Engaged to Formerly Engaged,…” This makes great sense to me.

“…which is a prediction of Value in the Future.” Hmm…I suppose, as I’ve stated my understanding thus far. I would need to run some tests to confirm my suspicions.

“So you can have Endorsers who are Formerly Engaged, you can have Low Value customers who are Highly Engaged – and buckets in between. To optimize the system, you would use a different Marketing strategy for each segment, which is the whole idea behind Relationship Marketing. You could then step up to REAN to implement Relationship Marketing based on the Customer LifeCycle.” You are defining tic marks on a scaling system, yes? Accepted. Different marketing strategies for different segments? Definitely yes.

REAN? Haven’t gotten that far in my readings yet. Sorry. Guess I’ll close for now and add more once I’ve studied a bit more. – Joseph

(responding to Jim Novo)

Howdy,
I realize quickly that one of my hopes for taking part in this discussion is being realized. Your suggestion to read http://blackbeak.conversionchronicles.com/2008/01/29/measuring-online-engagement-re-visited-and-introducing-the-rean-model/#comment-6761 was quite enlightening. From that post and elements in this thread (still going through it, folks) I form the belief that what I define as “engagement” is what must occur neuronically before what is being described as “engagement” in these posts can occur. Fascinating.

I must ponder this…semantic differences can be mathematically bridged (something our Language Engines are designed to do, so I’m a little familiar with how to do it).

Brad Berens gets a kick out of asking me questions because it’s fun to watch me go into a fugue state developing solutions.

Hmm…yes, I can understand how the bridge should be constructed. The model isn’t complex, merely rich in hierarchies…

[allow me to continue thinking on this while I respond to something else you wrote...]

I did not detect any resistance to the idea that if we are going to measure Engagement, that implies a relationship, which implies “value over time”, which drives towards the idea that “likelihood to continue” is what differentiates measuring Engagement from measuring Activity.

So Activity defines Value:

You can have Best Customers or Endorsers or whatever you want to call them for the environment you are in – your highest value participants. At the other end of Activity / Value, you can have Low Value Customers or Detractors or whatever you want to call them. Their Activity defines their Value.

I accept (for the present) what I believe falls from the above: “Engagement” is a determination of an individual’s likelihood to continue exchanging value with another individual over time.

Just so we’re all clear on how I actually frame this; “Engagement” is a determination of entity A’s willingness to continue exchanging value with entity B over time.

The reason for the reframe from “individual” to “entity” is to allow for non-biologics (such as websites) to be on the sides of the equation. Allowing for this change in construct gives me the ability to access some neuromathematics dealing with how humans conceptualize identity structures with which I’m already familiar.

The reason for the reframe from “likelihood” to “willingness” allows me to include a metric NextStage has used since its early days; Loyalty (please remember that what I mean is probably not what web analysts mean by the same word. See Usability Studies 101: Brand Loyalty). In many ways I sense that your use of “engagement” has elements of trans-temporal reafference in it (Branding in Online Video, Branding and Online Ad Placement). Trans-temporal reafference! My god, that would be a beautiful construct!

Oh, James, I love you! You’re making me think! (not a pretty thing. Ask my wife. Much worse than my dancing)

Somebody tell Jim Sterne we have another eMetrics presentation, “How to get visitors Engaged – Moving Loyalty from Their Minds to Your Wallet!” Or maybe Lars Johansson. He has a conference coming up, too, yes?

(Okay. I must calm down. Slow breaths. That would be a kicker presentation, though. A kind of “Here’s what has to happen in a visitor’s brain in order for them to produce value over time. Here’s what you have to do to your website in order for those things to happen in the visitor’s brain”)

I must adieu for a while and allow these thoughts to gel.

More to follow.

(and thanks for making me think)

And now at Jeff Chasin’s 3 Feb Comment…

Welcome to the FunHouse, Mr. Chasin.

And now at Jim Novo’s 3 Feb Comment…

“Did you test an optimal reactivation trigger point? In other words, how long would you allow them to be in the process of dis-Engagement before you tried to reactivate, and was there an optimal time frame in terms of response? Because typically, their likelihood to reactivate would decrease as time goes by…did you see any of that?”

And some folks think math is difficult to understand? (LOL!)

I had to read that paragraph three times to begin to get an idea of what is being discussed. It’s like listening to a foreign language; sometimes you hear sounds that you think are words but when you put them all together you still don’t know what’s being said.

But I’m willing to learn…

Now off to read Jim Novo’s suggested http://blog.jimnovo.com/2007/04/25/measuring-engagement/

(doing my best to get caught up, folks)

And now picking up at Mr. Jackson’s link, Measuring Online Engagement Re-visited and introducing the REAN model…

First, let me say “Yeah, Mr. Kaushik!” I agree completely that “…we shouldn’t use ill defined engagement metrics as a proxy for something solid like a sale.” The belief that we shouldn’t use ill-defined metrics is one of the defining beliefs of my research (and quite probably my life). Ill-defined metrics open us up to error, to misinterpretation, to various and ambiguous interpretations, … So, Avinash where ever you are, a nod to you, Good Sir!

One of the things I do in my research is test (repeatedly, ad nauseum and only until clients tell me to stop) how well defined a metric is. How do I do this? It’s kind of like surveying; you start with a metric that has repeatedly proven it’s level of accuracy then compare the metric under question to the known metric. Metric A is known to be accurate to within 2%. Metric B agrees with Metric A to within 5%. Metric C agrees with Metric B to within 7%. I can then say something like “Client, feel free to use Metric C. It’s not as accurate as B or A. In fact, the results could be off by as much as 70%. However, if you wish to use Metric C then so be it and let’s do our best to get the job done.”

Is Eric’s method or the reformulation of it “synthetic” because it’s comprised of several “basic” metrics? I wish my fellow Center for Semantic Excellence fellows were following this thread. Regarding the WAA definition of dimension, it’s a wonderful definition of “dimension” and if using it achieves some goal, then excellent and go for it.

You write “When you segment your metrics into these four dimensions then the semantic arguments around ‘what engagement is’ become irrelevant as the whole lifecycle is defined.” is excellent because it indicates a shift in the discussion from a definition of “engagement” to a definition of “lifecycle”. Fair enough, provided we recognize that the conversation has moved from a panthetic definition of “engagement” to a definition of “lifecycle”.

My reading of the explanation offered (without seeing a formula or calculation of what is being described) leads me to wonder, if everything in Eric’s formula fits into something in REAN, then

A) does everything in REAN fit into something in Eric’s formula? I’m curious because one of the goals of the reformulation was transmutability of variables.

B) does the final REAN calculation face some of the same challenges Eric’s formula had?

C) what limits are posed on the model by the collection and calculation methods?

D) (probably the big one) have you and Eric independently and simultaneously developed the same model and merely applied different names to it (much as Newton and Leibniz independently and simultaneously developed modern calculus)?

Let me offer at this point that I have no objections to whatever definition, whatever model and whatever calculation people want to use. My goal goes back to the beginning of this comment; I want to find Metric A, the one with the greatest accuracy that can be repeatedly proven to be accurate across the largest methods of measurement (or discover what the limits are on those measurements so that I can say “Given these conditions, Metric A will have this much accuracy”). From a thorough and complete understanding of Metric A I can then determine the relative values of Metric B, C and so on.

(and now going back to the comments made directly in this thread)

I agree that Attention, Engagement and Trust can be measured. NextStage does it all the time. We don’t do it with KPIs as I think you use the term, though, as we actually measure the behaviors that only occur when “…specific neural activity is taking place.” Same thing with engagement; we actually measure activity “…that serves to focus an individual’s Attention.” As our definitions our different, I accept that we’re measuring and reporting on different things.

I, too, can offer case studies if that’s useful. I don’t think it is as (to me) it serves to demonstrate what I’m discussing above; “given these conditions, Metric A will have this much accuracy.” I also recognize that case studies are use cases are demonstrations of a theory applied to the solution of a specific problem. (I do congratulate you on the successful application of your model, though.)

You write “To measure and analyze on and offline behavior and then try to predict who to market to by figuring out what they think is not doable with one tool or one metric.”

Well…umm…this confuses me a little. NextStage’s tools are all about how people think or how they will think. While they’re on a site (or whatever), interacting with marketing materials, etc. Often in real-time. I’m even offering a training on the subject in NYC in June that demonstrates how to put this kind of information into practical web and marketing designs without having to use NextStage’s tool sets to do it. I’d rather you use NextStage’s tools, of course, and it’s not necessary with some workarounds.

Your statement really causes a paradigm shift for me, though. Are the readers of this thread aware of the rich history “figuring out what they think” has in the social sciences? If not, go read Reading Virtual Minds or scan any of the bibliographies I’ve posted for my columns or Emetrics sessions. Go have a look at NextStage’s patent if you really want to cross your eyes. I have about 5,000 volumes in my personal reference library on the subject, everything from the Horst classic “Psychological Measurement and Prediction” to about fifty years on both sides. What NextStage brings to the table is a way to perform these measurements through any standard interface (hence one of the aspects of the reformulation of Eric’s calculation was to give it that same ability to migrate across interfaces).

Your statement is really causing me to wonder if it’s time to rethink our decision to grow organically via client referrals (thanks for pushing me to rethink). NextStage’s value proposition is that we do provide our clients with a richly detailed knowledge of how visitors are thinking while they’re navigating their website (for example). This richly detailed knowledge is translated into design changes that get visitors to think in ways more profitable for everyone involved (ie, “conversions”).

On one hand, I think APS (Association for Psychological Science and yes, I’m a member) folks would enjoy hearing “To measure and analyze on and offline behavior and then try to predict who to ‘xyz’ to by figuring out what they think is not doable with one tool or one metric.” On the other hand, let me share an anecdote from one of my mentors, a Nobel Laureate in Chemistry who has no degree in chemistry. His acceptance speech started with “I really don’t know why you’re giving me this. This problem would have been solved years ago if you had a physicist look at it.”

I find myself in some what the same situation reading your comment. It’s quite possible to accurately predict how people will respond and how they think, and how they’ll respond based on how they think. Progress Software even presented NextStage with an award because of our accuracy. Not sure if that qualifies as a case study and we have several examples along those lines, if they’re of interest.

You write “Most clients just ask questions like ‘tell me what my visitors are doing’.”

Yes, I agree. NextStage clients are using traditional analytics tools to answer those questions and we often refer them to analytics vendors and consultants if they don’t have such tools in place. The statement NextStage most often hears is a little different though, “Tell me what I should be doing.” A goal for me and one of the reasons for working with Eric on his calculation was to determine a) is it possible and b) how best to integrate NextStage’s, traditional web analytics and whatever else might come along into a comprehensive model. Now I know it is possible to integrate any number of analytic models into something more comprehensive, a boon for all concerned, me thinks.

To that end, I’ll be at Emetrics Toronto and SF along with various scientific and research conferences you can find listed on the NextStage site (haven’t listed them all yet, sorry) and can be had for a good cigar and an even better scotch. Let me know if you’d like to chat and I’ll set aside some time.

Thanks – Joseph

Now picking up with Eric’s “Given the direction our (…) industry is heading, I’m absolutely convinced that if we’re afraid to model and make more complex calculations, well, eventually we’ll pay the price.”

I have to admit, sometimes my time delay in getting to things can amuse me. Eric states in the above that, unless we make more complex calculations for our metrics models and Steve Jackson suggests that the model I came up with is too complex to be used by practitioners, consultants, etc.

Yes, well…back to the drawing board, I guess…

I do think the framework I supplied is as universal as it can be and I’ve shared some of the resulting formulations with Eric (I believe we’re going to be publishing them at the SF Emetrics. Is that accurate, Eric?). The big gain from the alternative formulations is that they allow for new and alternative metrics to be used when and as necessary.

Both Eric and I are busily throwing various data sets at the reformulation. Our definitions aren’t the same and they are complimentary. There are times it makes sense to use both as part of the same uber-metric, times it makes sense to use only one or the other, times it makes sense to mix and match elements of each with the other.

That was one of the points of the exercise (for me). Then again, one of my philosophies is that we are more together than any one of us can be apart. Too much social anthropology, probably. Sigh.

To Jeff Chasin: I guess I’d wonder if anyone is still paying attention. I could respond sooner and then I’d be sacrificing my desire for accuracy.

To Steve Jackson: The reformulation I gave Eric doesn’t handled “disengagement” so much as it recognizes an “engagement” scale (0-10, for example) that can be applied to a sales funnel (or X funnel taking into my account to create a unified metrics theory). This allows you to recognize that individual, some or all of your visitors are moving forward, static, backward, and can be tied to a variety of site and psychodynamic factors (depending on how complex/complete you’d like the calculation to be).

Your REAN – mobile user engagement is similar to one I’m familiar with. I accept that you were able to provide a usable metric to the client. I wonder about things like completeness of the metric provided, ability to segment and how fine a segmentation is possible. I also worry about providing clients with metrics that violate “conservation of units” concepts (not implying this is the case in your example, only stating a concern). I’ve encountered too many organizations relying on metrics that — when placed under the microscope — turn into something like David Guaspari’s great line about being able to tell if the fatness of a pig is more or less green than the designated hitter rule. Any calculation can come up with a number. The question is whether or not the calculation can come up with similar numbers when the variables translate to new interfaces that’s important.

For example, as I read your comment it seems you’ve applied your REAN model to mobile users. This is excellent as it demonstrates the ability to translate across interfaces. I’ll need to research that model before I can continue to comment, though.

God knows when I’ll post here again…

Howdy and thanks for your comments. I’ll do my best to respond to them.

Firstly. What are you trying to achieve? I’m working to create a framework that allows different disciplines to merge in ways that provide value. I’m pretty sure I’ve stated this several times and in several ways in several places.

what will it allow me to do as a consultant with my customers? or as a practioner? Before you answer that bear my second point in mind. I have no knowledge of you or your customers, hence am at a handicap and will do my best to explain what the formula allows me to do as a consultant with my customers and in my practice.

• This reformulation allows me to let my customers know which definition of “engagement” or which mixture of definitions is best going to suit their business objectives.
• I can determine which of all metrics is most directly affecting visitor “engagement”.
• I can determine what — out of all possible modifications — needs to be modified and how to modify it in order to achieve business goals.
• It shows me very quickly how different “engagement” paradigms come together and allows me to begin studying their overlap.
• From here I can begin predicting which visitors will be engaged and at what levels, possibly providing a new segmentation for the sales funnel. (might have commented on this in more detail at From TheFutureOf: 22 Mar 08 comment)

I find the above useful in my practice and none of them is as important (to me) as recognizing that the form shown above is the theory. Now that I understand the theory I can very easily determine which applications of the theory will work with browsers, with smartphones, with mall kiosks, … However, without first knowing the theory I could never fathom that a chosen method for determining “engagement” on a website would fail when attempting to measure the “engagement” of someone walking down the street looking for the nearest Starbuck’s on their smartphone.

Eric wrote at one point “The purported complexity of my calculation…” and went on to mention that some folks were having various challenges with it. One of the things that falls out of the above is three or four simplified versions of Eric’s calculation, each version adding a level of accuracy based on a) his definition of “engagement” and b) how much effort someone is willing to put in to determining how “engaged” visitors are.

This is quite valuable to me.

Secondly, showing the current example to my customers would result in me being showed the door and never being invited back. The average practitioner might be put off by it’s complexity. I would suggest this would be something that both NextStage and Eric would never try to explain to a client. At least when I’ve worked with Eric his thoughts are easily translated into non mathematical terms. I’m guessing that your clients, Eric’s clients and NextStage’s clients hire us for the same reason; to make the intractable tractable. Lots of NextStage’s clients come to us basically saying that web analytics doesn’t tell them what to do, only what has been done. One of the things NextStage does is come up with mathematical models that then become simple tools that provide simple yet concise instructions on what to do. This isn’t my attempt to sell anything here, it’s my attempt to share that without a thorough theoretical understanding then applications become vertical boundaries. Theoretical frameworks provide horizontal frameworks from which a wide variety of vertical applications can grow. And better, by knowing what doesn’t work in application A I can quickly determine whether or not something similar might work or not in application B.

Example: A hammer is a hammer is a hammer, and until you understand the concepts of force over distance and basic angular momentum principles (you don’t need to know those terms, only the principles behind them) then you’ll never understand that knowing why you should drive a nail by holding the hammer at the base rather than the neck explains how to hit a ball out of the park every time you swing (and connect, of course).

(And it’s probably worth noting that I am what is considered a “second-order” tool maker. I create tools that create tools.)

I guess my question is how do we ever communicate this easily when this is the way forward?

SELECT “hi(vi)” AS “ENGAGEMENT”
FROM “N – n”
WHERE “D[hk(vk)]” = somevalue
AND “D[hj(vj)]” = someothervalue
AND …
GROUP BY “xi – ti”
ORDER BY “V” (or “I” or even “A”)

The error is mine as I thought framing the formula as a SQL statement would have explained it a bit.

If this is an academic debate … I disagree that it’s an academic debate. That’s just me.

…that will explain to the minority who read this blog and understand your lines of thought (me included by the way) that there is a standard way to describe and understand engagement…yes. That’s the key. There is a standard way to describe “engagement” based on how the term is defined. It’s not a matter of correct or incorrect descriptions because (like relativity) the best definition depends on a) what you want to measure and b) how you want to measure it.

…then I’m for it because it allows me to see how the great minds in this field are thinking.

However I would never use it unless I could explain it to a 55 year old executive that has an extremely busy schedule, has never done web/customer analytics and doesn’t particularly care about the math, but does care about how to spend the budget she has for this quarter. As you wish (and using some minor modifications to Eric’s definitions to simplify the calculation); “If you, Ms. Executive, can provide me with click depth, loyalty, feedback and interaction values then I can provide you with a reasonably good idea of how ‘engaged’ the majority of your visitors are. If you can add in duration and recency then I can both blow your socks off and tell you what to do to get more of them engaged. If you can segment your audience into branded and non-branded events then I can tell you how to get more conversions. If you can provide the average number of visits it takes for a visitor to convert, we can fine tune further, possibly shortening the interval.”

However, without going through the math I did I would never be able to suggest a statement such as the one above because I wouldn’t have noticed which metrics (click depth, loyalty, feedback and interaction) have common factors, how to redefine those metrics so that they generate (depending on your requirements) either smooth curves or step-wise functions, that duration and recency require different scaling functions to behave correctly in the calculation or that branded and non-branded events are providing completely different information sets about site visitors.

I don’t see the point of creating anything which can’t be easily described and understood by everyone who sees it. That’s why I find this a tad irritating. If you can boil it down for my customers I could get really enthusiastic about it. Fair enough. My role in most of my jobs has always been to provide technical or research foundations for others to use in their practice so what I’m offering here might not be of immediate value to you.

Yes we may have some long and convulted equation that works for describing engagement but unless you can boil it down to something really simple like e = MC squared or better yet engagement equals click depth and duration then the people that matter (the clients paying our salaries) aren’t going to be impressed by any of this work. I’ve been likening the above to general relativity and Eric’s calculation (corrected) to special relativity in discussions for a while now.

We all have different definitions of engagement now and as I understand it translate that to the customer quite well.

Apart from attempting to define something not currently defined in mathematical terms what does this discussion achieve? I think I explain this above. Let me know if I don’t.

I mentioned before that Eric had published what I thought was a very comprehensive formula to describe engagement. I said that because his formula touched on every aspect of the customer lifecycle, it measured an aspect of the whole thing. But Eric freely admits that it wasn’t the answer to everyones’ engagement problem, merely an iteration of his own thoughts on the matter which span back years and allow him to determine his best source of leads from his own marketing initiatives. And with all due respect to Eric and you, the caveating that occurs in this paragraph is exactly why I went through the exercise. A definition of anything that only works for certain things at certain times under certain conditions is not a useful definition to me as I can’t guarantee that things, times and conditions will always apply. However, understanding why certain things, times and conditions make things work in certain ways I can translate the things, times and conditions so they’ll work in other ways as needed.

I’ve written before about the anthropologist and the microbiologist having lunch, the microbiologist looks at her watch and exclaims, “My goodness! I have to go destroy a culture!” and the anthropologist has a heart attack. Unless you recognize at what “culture” means then it’s hard to appreciate that two disparate fields could use the term with equal eloquency to describe different things.

Your version seems to be an even more in depth one than Erics. However it’s not in my opinion going to be the answer because it’s not going to be used easily.

I don’t have a definitive way to describe engagement, nor do I have a one rules all formula but I do have a way to describe the lifecycle and measure it in a client friendly understandable way. We call it REAN. Reach, Engage, Activate and Nurture where Engage is simply click depth and duration – in other words interaction metrics. For what it’s worth, the above formulation accomodates your definition of “engagement”. As I wrote in From TheFutureOf: 22 Mar 08 comment, “So, are simple metrics enough to define engagement? Depends how accurately you want to define it. If you want to use “session duration” to measure engagement then just call it “session duration” and keep things in their simplest form (the KISS philosophy). You have high session duration and want to call it engagement? Then great! The majority of your visitors are engaged. Are the majority of your visitors doing something useful? To themselves or to you? No? Then they are not engaged in an economically useful way (and I’m using “economics” in the NextStage sense of exchange, not a simple money concept). Eric’s definition concludes with the concept of business goals. If simple metrics fulfill your business goals then you’re good. If they don’t, join the discussion.

Howdy,

First, I didn’t forget the comments here. I got a little involved in some projects, and probably some of you know by know that I’m working with Eric on The Engagement Project (see Measuring Engagement Online: The Next Stage, Measuring Online Engagement: Step One and From TheFutureOf: What does the equation look like? for more on this).

The reason I stopped commenting here for a bit is because I stopped my comment stream at Eric’s “The purported complexity of my calculation…” and, as I had never looked at Eric’s calculation and he was entering it into the discussion, I decided I needed to study his calculation in order to understand the statements being made.

Anyway, the end result of that can be followed in different threads, some here, some in Eric’s blogs, some in other TheFutureOf posts and also on BizmediaScience (see Eric Peterson’s Engagement Project and the Engagement Equation, Part 1 and Eric Peterson’s Engagement Project and the Engagement Equation, Part 1 – Responding to WindKiller’s Comments for starters).

So, picking up with Eric’s “The purported complexity of my calculation accounts for that. Maybe the visitor is clicking but not paying attention, maybe they’re staying on the site, but again not paying attention. But are they interacting? Are they coming back to the site? Are they subscribing? Etc.”

One of the things that came out of working with Eric on his calculation was a quantitative recognition that our definitions of “engagement” were different because we were framing our definitions on different interfaces. Eric’s frame of reference is the browser and within that frame his definition works well. My definition’s frame of reference is the brain-mind and within that frame of reference NextStage’s definition works well. NextStage’s analytics are using scripting and related tools to measure what’s happening in a visitor’s psyche, Eric is using scripting and related tools to measure what’s happening at the browser.

Another thing that falls from this hearkens back to my original thoughts in this thread about A Meeting of Minds. The work we’ve done on understanding these different frames has produced a method for merging and mixing Eric’s definition, NextStage’s definition, anybody’s definition in ways that produce actionable results.

Yeah for our team, huh?

Another yeah for our team is that Eric’s calculation is no longer complex and is highly simplified. One more yeah for our team is that we can do things like use one reference frame to determine value in others (borrowing from physics a bit). NextStage has (at this point) about 15 years of online data and some 70,000 data points that it uses when making calculations about such things as attention, engagement, authority, trust, etc. It’ll be interesting to learn how this type of analytics merges with more traditional analytics environments (one of my goals this year).

So, is the visitor clicking but not paying attention? From both a psycho-cognitive and web analytics perspective, now we’ll know. Are they staying on the site but not paying attention? Ditto. Are they interacting? We’re able to not only say yes or no but also give good approximations of how much of their cognitive resources they’re devoting to your website (are they navigating the site while talking on the phone? Are they talking to someone sitting beside them while sharing something about the site? Are they navigating your site because someone suggested it to them? How important is your information to them? Answering questions like this is quite simple now).

More to the point, in answer to Eric’s “I think if they’re not engaged, the answer to those questions will be ‘no’ and thusly their calculated metric would be lower.” is a determination of just how engaged visitors en masse and individually are. This, in turn, means we can bring some more sophisticated mathematical tools forward to answer some questions such as:

1) How engaged does someone have to be before they convert?

2) How much attention does someone have to devote to a site during a visit in order to guarantee a return visit?

(and most significantly)

3) What can be changed/edited/modified/updated to make these things happen faster/sooner/quicker/better?

Eric also comments that people struggle with the calculation for various reasons. Part of the process we went through was to make the calculation more universal. The form shown in From TheFutureOf: What does the equation look like? is like general relativity. It’s meant to work under all conditions. Eric’s calculation (with some simple modifications) is like special relativity. It’s meant to work with metrics available through a standard and commonly used web interface and for his definition of engagement. To the question, “Is Eric’s definition valid?” Let me first offer that the economic value of a metric is directly proportional to:

1) the information value of what the metric reports on

2) as that information value is defined by some group with an interest in it and

3) that same group’s ability to change environmental factors so that

4) the metric changes report value (not information value) in direct and obvious response to that same group’s intentional changes in environmental factors.

In this context (and I admit it borrows from a study of language and semantics a bit), Eric’s definition is valid if consumers find value in it. The same is true for NextStage’s definition and everyone else’s.

And let me now share something that’s most important. Stating that engagement doesn’t exist as a metric is also valid if a certain group of consumers find no value in any existing definitions of engagement. It’s a question of reference frames and if no value exists for engagement in some reference frame, so be it. “General Relativity” will accept that and return (duh!) a value of “0″ for that reference frame.

Are simple metrics like “session duration”, “recency”, …, alone enough to define engagement? As I offer in From TheFutureOf: What does the equation look like?:

“Accuracy is a function of target size, not mathematical rigor. Accuracy of 10% with three variables active can quickly rise to 90% accuracy with as few as four or five variables active. Let me give you a “marketing” example. You’re selling to a) 53yo b) white c) males and you’re capturing 10% of that market. But what if you’re selling to a) 53 yo b) white c) males in d) NH with who e) are business travelers? Ah, well, now perhaps you’re capturing 90% of the market.

Some people aren’t aware that the opposite can also be true; it’s possible to achieve (for example) 90% accuracy with three variables and dwindle it to 10% when more variables are present. Imagine a bullseye style dartboard. You can get lots of darts in the yellow and good for you; that’s high accuracy. Then again, there are only five colors you can hit (five variables in the equation).

Now imagine a more traditional dartboard with a very small center area and lots of other areas indicating different values and multipliers. Both types of dartboards are circles, yet add or change a few variables and accuracy as a percentage of dead-centers is shot to heck.”

So, are simple metrics enough to define engagement? Depends how accurately you want to define it. If you want to use “session duration” to measure engagement then just call it “session duration” and keep things in their simplest form (the KISS philosophy). You have high session duration and want to call it engagement? Then great! The majority of your visitors are engaged. Are the majority of your visitors doing something useful? To themselves or to you? No? Then they are not engaged in an economically useful way (and I’m using “economics” in the NextStage sense of exchange, not a simple money concept). Eric’s definition concludes with the concept of business goals. If simple metrics fulfill your business goals then you’re good. If they don’t, join the discussion.

(more to follow, picking up with Eric’s “…eventually we’ll pay the price.” statement.

I think this entry will conclude my dissection of Eric’s “Engagement” equation and provide ample opportunity for others to investigate its properties in a cleaner framework. I’m going to start by offering that this entry will deal with Eric’s definition of “engagement” and finding a way to make it more comprehensive. Throughout this analysis I’ve been reminded of Nobel Laureate Murray Gell-Mann’s statement, “I have paid a certain fee to the word ’sustainability’ to make it mean whatever I want it to.”

In this case, lots of groups have paid certain fees to different definitions of the word “engagement” (NextStage included). I’d like to paraphrase something I wrote in my first post to this blog and that I said to Eric when I started investigating his formula several weeks back, “We’re all coming at this from our individual frames of knowledge. Within our individual frames, each of our definitions is correct. But in the real world our individual frames have very fuzzy, very inexact and often non-existent boundaries. It’s time to come up with a meta-frame in which all definitions have equal merit and can be mixed and blended as business needs dictate to produce real economic value.”

No, really, I said something pretty much just like that. Eric often tells me he wishes I spoke Engslich. I mean “English”.

Once again, Eric’s equation:

And now, Eric’s definitions (with a little cutting and pasting on my part):

• Click-Depth Index (Ci) is the percent of sessions having more than “n” page views divided by all sessions.

• Recency Index (Ri) is the percent of sessions having more than “n” page views that occurred in the past “n” weeks divided by Average Days Since Last Session (summed over all visitor sessions in the timeframe under examination). The Recency Index captures recent sessions that were also deep enough to be measured in the Click-Depth Index. The denominator is calculated using the timestamp of the current session being evaluated and the most recent previous session. In the data set today it appears to range between “0″ (meaning “no previous sessions associated with this cookie”) and some larger number (theoretically as large as the total number of days in the timeframe under examination.).

Ri has an interesting behavior: if someone has multiple sessions in a single day, the denominator is between 0 and 1, and Ri itself becomes quite large relative to the other calculations in the framework. I’ve asked to see if I can limit the denominator to be no lower than “1″ (save “0″ for first session) but am not sure that the behavior I’m seeing isn’t “good” (or “right” or however you would think about the behavior of a calculation … that’s your thing.).

• Duration Index (Di) is the percent of sessions longer than “n” minutes divided by all sessions.

The Click-Depth, Recency, and Duration indices are all pretty straight forward and are more-or-less the traditional indicators that most people (incorrectly) call “measures of engagement”. Each of these is very important to the overall calculation, but none of these alone are sufficiently robust to describe “engaged” visitors. I set the “n” values for my site’s calculation based on the average value for each and this seems to work pretty well (meaning my Ci looks for sessions more than “5 page views” in depth, my Ri looks for sessions more than “5 page views” that occurred in the “past three weeks” and my Di is looking for sessions longer than about “5 minutes” in length.)

• Brand Index (Bi) is the percent of sessions that either begin directly (i.e., have no referring URL) or are initiated by an external search for a “branded” term divided by all sessions (see additional explanation below).

Brand Index is a little more complicated. Here I have made a list of all the terms I believe to be “branded” for my site and business, terms like eric t. peterson, web analytics demystified, web site measurement hacks, web analytics wednesday, and the big book of key performance indicators. Whenever a session begins either with no referring domain or comes from a search engine with one of these terms attached, I count this as a “branded session” and score appropriately. While this index perhaps unfairly weights towards search engines, I firmly believe that if you’re starting your session with either my branded URL, my name, or the name of one of my books that you are already engaged.

• Feedback Index (Fi) is the percent of sessions where the visitor gave direct feedback via a Voice of Customer technology like ForeSee Results or OpinionLab divided by all sessions (see additional explanation below).

Feedback Index is the sole qualitative input to this model but it can easily be expanded if necessary. Here I am simply scoring sessions based on whether visitors are providing qualitative feedback via the OpinionLab “O” present throughout my web site or writing me directly by clicking a “mailto:” link. I’m not looking at whether the feedback is positive or negative, only whether feedback was given, operating under the belief that anyone willing to provide direct feedback is engaged.

The Feedback Index could easily be expanded by scoring based on the answer to direct questions posed to the visitor, questions like “do you find the content on this site valuable?”, “do you plan on calling Web Analytics Demystified about consulting?” and “would you described yourself as engaged with this site?” Given a sufficiently robust mechanism for making the calculation, the Feedback Index can provide a tremendously powerful input to the visitor engagement model.

• Interaction Index (Ii) is the percent of sessions where the visitor completed one of any specific, tracked events divided by all sessions (see additional explanation below).

The Interaction Index captures sessions in which specific “engaged events” occur other than the site’s primary conversion event — events like downloading a white paper, providing an email address, requesting a presentation or PDF, commenting on a blog post, Digging a post, emailing content to a friend, printing a page, etc. The Interaction Index is designed to capture a small weighting from those measurable goals on your site you believe to be indicative of engagement.

The Interaction Index specifically does not examine commerce transactions and other conversion events of fundamental import to the site. While I have debated this in the past, here is the rationale for recommending the exclusion of primary conversion events:
1. These events already have their own key performance indicator: conversion. Given that conversion is likely already defined for most transactional sites and tracked in great detail, adding conversion to the visitor engagement calculation is superfluous in my opinion.
2. The visitor engagement metric is designed to provide information about the large number of visitors who do not convert. Given relatively low conversion rates online, having visitor engagement be decoupled from conversion provides a cleaner measure for use in exploring non-purchaser behavior, including looking for independent correlation between the two measures.
3. By excluding conversion, the two metrics can be used side-by-side to look for visitor behaviors may not be obvious otherwise. Given the lifetime of possible visitor behaviors, having a way to look for well-engaged visitors who have not completed a transaction online or have completed a transaction outside of the available data set provides a critical view not otherwise readily attained.

In addition to the session-based indices, I have added two small, binary weighting factors based on visitor behavior:

• Loyalty Index (Li) is scored as “1″ if the visitor has come to the site more than “n” times during the time-frame under examination (and otherwise scored “0″)
The Loyalty Index is a reflection of my belief that repeat visitation behavior is perhaps the best measure of engagement available. Based on the distribution of visitor loyalty data at Web Analytics Demystified, I score “1″ when visitors have come to the site more than five times in the past 12 months.
• Subscription Index (Si) is scored as “1″ if the visitor is a known content subscriber (i.e., subscribed to my blog) during the time-frame under examination (and otherwise scored “0″).

The Subscription Index is a reflection that truly engaged visitors are able to self-identify by subscribing to our blogs or newsletters; if you have taken the time to subscribe to one of the Web Analytics Demystified blogs I believe you to be engaged. If your site does not have some type of XML-based content subscription you can either drop this index or (perhaps better) look for an opportunity to develop a subscription service, thusly giving your visitors another good engagement point.

You take the value of each of the component indices, sum them, and then divide by “8″ (the total number of indices in my model) to get a very clean value between “0″ and “1″ that is easily converted to a percentage. Given sufficient robust technology, you can then segment against the calculated value, build super-useful KPIs like “percent highly-engaged visitors” and add the engagement metric to the reports you’re already running.

And now Eric’s definition, “Engagement is an estimate of the degree and depth of visitor interaction on the site against a clearly defined set of goals.”

Eric has provided us with the A-space form (the definition) – “Engagement is an estimate of the degree and depth of visitor interaction”. As an interesting side note, compare Eric’s definition with NextStage’s, “Engagement is the demonstration of Attention via psychomotor activity that serves to focus an individual’s Attention.” Hmm… I appreciate that it might not be obvious at first glance and these two definitions have much in common even though their original frames are very different.
Eric also provides us with the interfaces his definition applies to, “…on the site…”.

Thus the interfaces can range from a traditional and commonly used web browser to a smartphone to I don’t know what. Here I will throw in some NextStage research to clarify things a bit; How people “browse” a site on a smartphone is very different than how they browse a site when sitting in front of a computer. This is what I was referencing when I was being so niggly about different interfaces. To NextStage, “engagement” gets measured differently due to the different interfaces and I borrowed that concept here to simplify things a bit.

Let’s focus on computer-based browsers as the interface for now. This frees us to explore the variables Eric used more easily. Note that I’ve made some minor modifications to Eric’s variables to provide for something called “conservation of units”. Let me give you an example of “conservation of units” with some typical questions we use in our trainings:

Which uses more water, a shower or a curtain?
If something is empty, is it cold?
Do you walk to school or carry your lunch?
What’s thinner than coffee?
Which is lighter, a car or an automobile?
How young is a teenage adolescent?
Is “none” the past tense of “noun”?

These and similar questions are designed to cause fugue states in individuals without proper training because they make sense at one level and not at another. The level at which they don’t make sense is where units aren’t conserved. “Empty”, for example, is a different metric from “cold”, “coffee” uses a different metric than “thin”. It’s kind of like asking someone “How long have you been at this?” and they respond “Oh, about four pounds.” The answer makes no sense in response to the question (unless you’re at a gym, perhaps). Back to Eric’s metrics:

Ci == is the percent of sessions having more than “n” page views divided by all sessions for some given period of time, t, divided by all sessions regardless of number of page views for that same period of time, t.

In the above, P is “Page Views” and S is “Sessions”. There is a very good (to me) reason that I place the time constraint, “…for some given period of time, t, …” in the above calculation and I freely admit it comes from something NextStage gets asked a lot, the ability to differentiate results based on some client activity such as launching a new campaign.

Using this same methodology, Ri becomes that shown on the right.

A concern in the definition of Ri is that it allows for some division-by-0 blowups. This is alleviated by separating the time period being investigated, t, from the time since last session, ΔT1. The above makes a few changes to Eric’s definition. The time, T1, is always less than or equal to the time period being investigated, t.

In other words, if you’re investigating visitor “engagement” over a day’s period, t == “day” and T == “hours”, if t == “hours” then T == “minutes”, and yes, I am playing a little fast and loose in this paragraph and not in the definition.

Also, Eric has “…(summed over all visitor sessions in the timeframe under examination).” This is actually handled in the summation over all visitors and doesn’t need to be repeated here hence is not included in the formula above.

Di takes on a similar form to Ci above and follows similar logic.

Again, I’m suggesting a minor modification to Eric’s definition to conserve units. The modification is that dz be all sessions with a measurable duration both within the given timeframe and of similar time unit to that used in Ri rather than simply all sessions.

The Bi metric, as Eric defines it, is…umm…interesting. In many ways. Eric writes “…if you’re starting your session with…you are already engaged.” I’m going to suggest that phrasing be removed because including it means the entire framework becomes something like “I know you’re engaged because you’re engaged.” Eric’s statement is actually bringing another interface, the human brain-mind, into the system and that would make the formula a little more complicated than we need at present. At some point and if there’s interest I can share how complimentary metrics can be applied to both interfaces to provide a richer, more robust framework.

There are some simple and obvious solutions to this that don’t involve things like modality engineering and the calculus of consciousness. Right now we’re developing a formula that has the form “E = …”. We can have a second formula, “E(B) = …”, in which all visitors in the summation are constrained by both some time period, t, and that they are branded, “(B)”, then test for differences. This would address the “your engaged because you’re engaged” confusion (not to mention really proving if branding and search make a difference). In addition and based on our starting conjectures, you could have something like “E = E(~B) + E(B)”.

At present, therefore, my suggestion would be to go with “E = E(~B) + E(B)” because (in my opinion) it removes any elements of confusion about cyclical logic or definitions invalidating the formula.

Like Bi, Fi is also interesting in many ways. So long as the metric only deals with whether or not a visitor provided feedback and doesn’t take into account the methodology of that feedback then it has a form similar to those above

Note also that this is the first duplicated index we’ve encountered. The “mn” and index from 1 to N there are ways to simplify the final calculation further should that be of interest. I will also offer one thought regarding Eric’s “The Feedback Index could easily be expanded…”; Like Branding, this is a calculation that can become much richer and more robust with complimentary frameworks in place.

Ii suffers from the same problem as Bi due to “…engaged events…” and is another example of “You’re engaged because you’re engaged”. I also think this metric suffers from an attempt to synthetically apply a brain-mind interface to the formula. Who decides what an “engaged event” is? Why does this event indicate engagement and that one not? The fact that similar events have conversion as their KPI forces “I” type events to be some kind of “soft conversions”. Are all events that don’t lead to a sale “I” type events?

The greatest concern with the existing definition of this is that by separating “I” type events from “conversions” we create a discontinuity in the sales cycle. The goal (I think) is to get people so “engaged” they convert and if “I” type events are distinct from conversion events then we don’t learn the tipping point, per se, and the ability to isolate the one (or more) elements that need to be adjusted in order to increase conversions is lost. I’ll point to Usability Studies 101: Defining Visitor Action as suggested reading for this.

Again, I believe there’s a simple solution for this; a step scale for events required for a “conversion”. For example:

Event Designation	Metricable Event	Value
A	Landing Page	0
B	Navigates site	1
C	A+B	2
D	Navigates Product Path	3
E	A+D	4
F	Requests Email Follow Up	5
G	C+F	6
H	E+F	7

What this becomes (progressing from what I’ve demonstrated here) is a sales funnel (see Usability Studies 101: The X Funnel). Eric writes “…is designed to provide information about the large number of visitors who don’t convert.” and that’s a valid goal. Remembering that the original summation is over all visitors, this methodology provides a clue as to where the great unwashed are in the sales cycle, ie, how “engaged” the majority of visitors are. Now you have a basis to determine where to start your (for example) your A/B testing without burning through your research dollars on a whole site or entire page. Another, more obvious benefit from the above concept is that it allows Eric’s I_i metric to comply with our conservation of units principles from before.

Again we see some simplification becoming available down the road.

L_i as currently defined fails to meet the formula base requirements for several reasons, although primarily it is a variation of the “you’re engaged if you’re engaged” confusion. IE, who decides what “n” means someone is “engaged”. Fortunately a solution to this was already demonstrated above with the reformulation of Eric’s I_i metric. The L_i metric’s table is a variation on a straight count of visitor sessions as shown below

# of individual visitor’s sessions	Value
1	0
2	1
3	2
4	3
5	4
6	5
7	6

The beauty in this rethinking of Li is that you can use the above to create version A and, with a little modification, version B

Version A is cleaner in the inclusatory sense because it continues with the simplification of the final equation.

Version B offers the ability to recognize the tipping point from loyalty to disloyalty, if you will, or for the purposes of Eric’s formula when “engagement” becomes “disengagement” because “L” is the average number of visits for all visitors within the given time frame, t, that participate in a “conversion” event. Note that the event doesn’t have to be “conversion”, it can be any event or group of events you want to query. For example, if you have 50 conversions in some time period and the average number of visits per converted visitor is 5 but your average number of visits per visitor is 10 and you’ve had 500 visitor sessions during that period? It’s probably time to investigate those non-converting visitors. They’re doing something and despite all other metrics, it’s something to pay attention to.

This leaves us with Eric’s S_i metric. I’ll offer without proof (I’ll do it if you want me to and forgive me, I think it’s obvious) that Eric’s S_i metric is subsumed by the rewritten I_i metric.

Thus a final form for Eric’s formula is

Note that if you’re not considering “Branding” events then the second term becomes “0″. For that matter you can lop off any of the terms you want if you recognize that this is an example of increased accuracy coming with increased variables.

What else what else what else? The form above looks different from that posted on From TheFutureOf: What does the equation look like? because the latter is (more or less) general relativity and the above is special relativity. The latter assumes all possible inputs and generates all possible outcomes (and the things you can derive from it are freakin’ amazing). The above is the form general relativity takes when you use Eric’s definition of “engagement” on a website.

Okay. I’m done.