Title	Details
Role	Lead Crative Technologist
Tools	Unity, DMX, MadMapper, Spherical Projecting Touchscreen, Speech To Text, Physical Computing, Arduino, Visual Display Solution
Length	2 months
Project	Eight interactive “hives” were created to showcase Accenture’s vision for the future of eight business categories, including Smart Living, Finance, Cloud, 5G, Human Potential, New Consumer, Digital Health, and Life Sciences.
Impact	Deployed two out of the eight experiences successfully
Collaborators	Future Colossal

Role

At the heart of Accenture’s New York headquarters in Hudson Yards lies a captivating display of their thought leadership, consulting expertise, and industry experience.

I worked at Future Colossal as Creative Technologist for two of the eight hives to help showcase Accenture’s vision for the future of eight business categories: Smart Living, Finance, Cloud, 5G, Human Potential, New Consumer, Digital Health, and Life Sciences. These hives utilize cutting-edge technology, stunning art and lighting, and engaging storytelling to bring Accenture’s research to life. This project helps to highlight Accenture’s commitment to shaping the future through innovative thinking and digital transformation. All eight hives were designed, produced, and installed during COVID-19 quarantine restrictions.

Demo

Innovation Hives

Smart Living

Project Mission

The installation showcases Accenture’s sustainability efforts through a spherical touchscreen display, a custom fabricated steel map of the NYC, and two 65” TV screens. Users can interact with the globe to learn about Accenture’s impact on communities. Upon interaction, dynamic lighting of the LED lights behind the custom steel map animate corresponding to the specific touch points. Heat maps and data visualization would display accordingly dependent on the

Solution 

The project’s mission is to educate and raise awareness about Accenture’s efforts in these areas. Our installation showcases Accenture’s commitment to sustainability and global inclusiveness through interactive displays. The Pufferfish globe invites visitors to engage and learn more, triggering a multimedia experience across two TVs and an NYC steel grid with LED strips.

How we accomplish it 

We used a Pufferfish Globe to send UDP messages to the main application, two TVs, and a steel grid with LED strips controlled by Advatek and Madmapper, to showcase Accenture’s sustainability and inclusiveness achievements in an interactive way.

When a user interacts with the pufferfish globe, it sends out a UDP message for Unity to receive. Although several people can interact with the globe at the same time, or the globe takes more than one touchpoint, Unity would receive the message and display the corresponding media content individually in a timely fashion. There is no queue. When one touches another hotspot right after the previous one, the Unity app will display the second one by cutting off the content of the first one.

Worked with Pufferfish company

I worked with Pufferfish company and learned how to use the Pufferfish within a week and also wrote a script to send out UDP message from Pufferfish and was successfully received by the Unity application. I later documented the whole process for the company for internal purpose to save training hours.

One challenge I encounter was we wanted our button to be animated video with transparent background. We tried different video types with transparent background, they didn’t display through the Pufferfish Application Builder, the app to customize the design for the 360 projected touch-screen globe. I then used ffmpeg to experiment with the HAP Codec for the video and they finally worked.

Led the project

I led the project and delivered the project within 2 months. I made the MVP within 1 month and refined from client’s feedback within 1 month.

I designed and implemented logic for the following features:

• Communication protocol to receive UDP from Pufferfish and send UDP message to the Artnet Lighting system
• Heatmap shader from real time traffic API
• Spout out render texture to MadMapper for light mapping for the Artnet Lighting System
• Asset storage, calibration, and design transitions for agile onsite installation

Finance Forward

Project Mission 

“Finance Forward” offers a unique insight into the ongoing digital transformation in the financial services industry. Through AI-controlled voice and interactive rotating dial, it presents the latest trends in finance, featuring constantly updated information from Wall Street and historic vintage news. Experience the future and history of finance in a captivating, immersive environment. This project drew inspiration from stock ticker installation in a modern structure to display how the core areas of the American financial system were rapidly changing.

User Experience 

Our experience transports guests through the digital transformation of the financial services industry, from the past to the present and future. The Idle State displays up-to-date stock graphs, tickers, and a dial that remains off. The Interactive and Deep States are activated by either guest interaction with the dial or docent control with Collaborative Voices AI technology. Guests can experience different narrative themes and information that dynamically transform the space with a turn of the dial or a voice command. The color of each theme is unique, creating a fully immersive experience.

Solution 

Our solution involved integrating a combination of stock tickers, multiple monitors, voice AI technology (UnityEngine.Windows.Speech, Offline Recognition Plugin), microphone and audio interface, and a MSI PC. This allowed us to deliver a dynamic and interactive experience, utilizing real-time data and voice activation for seamless content presentation.

This project includes this Main Application, MadMapper that controls the Tickers and 10 screens:

• Arduino Technology Solution with Powermate dial
• Voice Technology Solution
• Visual Display System: Tickers + 10 screens

What I did

Idle State

This application displays the latest stock market data from TD Ameritrade API in the form of graphs and charts. The tickers showcase the ticker information of the relevant stock in horizontally scrolling motion.

Interaction State

Since guests can choose a variety of options to display the stakeholder-facing content, I wrote a category manager to store all the relevant info in terms of the name of the topic, the index of the topic, and the trigger method (by dial or by voice). This helps me to display the correct video for the corresponding category.

• Controlled by Dial

I coded to receive the UDP message from Arduino in order to determine the asset being displayed as a whole.

• Controlled by Voice

I integrated the Microsoft Speech Recognition system to listen to the keywords for the categories to trigger the correct video for the corresponding category. We also censored out all potential words that are not safe for work in our algorithm.

Final Mapping

We utilized the MadMapper to do the final mapping to match to the screen locations and do onsite calibration.

Cloud

We developed interactive, free-standing screens to demonstrate the advantages of adopting cloud technology across a company. The content, outlining six distinct benefits, was spread across six screens. Human-presence sensors on these screens activate animations illustrating these cloud benefits. Visitors are visually represented as point clouds, effectively showcasing essential metrics and advantages of Accenture’s Cloud services. This setup serves as an engaging introduction, guiding guests through a story-driven path toward a workshop area, seamlessly connecting the experience to a hands-on demonstration space.

Human Potential

The pandemic has significantly altered traditional work models, prompting Accenture to spearhead the transition towards digital work environments. This shift highlights the importance of recruiting, engaging, and retaining a diverse workforce from various global locations. To showcase this transformation, we designed an experience that demonstrates the widespread impact individuals can have, regardless of their location. Utilizing a “voronoi” algorithm, we visualized the workforce as a honeycomb-like structure of cells. Through position tracking sensors, we illustrated how guests’ movements generate dynamic cells within this honeycomb, creating ripples that symbolize their influence on the broader network.

Digital Health

The Hive, created to clear up confusion and misinformation in healthcare, features a digital fog that reacts to user movements. When a user pauses, the fog clears, unveiling brand messages and their reflection. Led by Accenture’s Healthcare Assistant, guests explore industry innovations.

Life Science

Accenture’s exhibit showcases their advancements in data and health technology, transforming biopharma and patient care. Set in a lab-themed space with an LED art installation, it features dynamic DNA and molecular animations on LED “test tubes” that shift from colorful chaos to clear images. Animated videos, triggered by a docent, highlight Accenture’s New Science initiatives across screens. Guests interact with these screens, while a custom app enables the docent to manage the display content for an integrated experience.

New Consumer

The Hive showcases future retail trends by inviting guests to interact with custom 3D-printed objects on a museum-like shelf. Interaction with any of the eight “Smart Objects” triggers specific content via RFID sensors, while the other eight “Virtual Objects” not only trigger content but also display a 3D on-screen replica that moves and rotates in sync with the physical object, highlighting the type of virtual interactivity used in modern retail spaces.

5G

We created an Augmented Reality (AR) “window” that transforms the static Manhattan skyline seen from Accenture’s offices into an interactive display, illustrating the impact of 5G on daily life. This self-sufficient, scalable AR technology lets users manipulate maps and view animations showing 5G’s effects on various locations. Equipped with a high-definition hidden camera, this touch-screen display also presents real-time data on NYC’s transportation system and highlights other technologies enhanced by 5G.