Augmented Museum

Augmented Reality application to display paintings on a surface.
Explore the docs »

View Demo · Report Bug · Request Feature

Table of Contents

1. About The Project
   • Built With
2. Getting Started
   • Prerequisites
   • Installation
3. Usage
4. Roadmap
5. Contributing
6. License
7. Contact
8. Acknowledgments

About The Project

The idea of this project is to give a user the ability to create a virtual museum in whatever physical space they have (home, office, etc.). The goal is to create framed paintings/photos to be "hung" on a wall as one would normally do.

Presentation video and demo can be found here!

IEEE Conference Write up can be found here

(back to top)

Built With

• C++
• OpenCV
• CmdParser Library

(back to top)

Getting Started

To get a local copy up and running follow these simple example steps. This application was built and tested on a Mac OS using g++ and Make to compile the code.

Prerequisites

• Make
• OpenCV

Installation

1. Clone the repo

git clone https://github.com/Kevin-Heleodoro/Augmented-Museum.git

2. Run Make from the root directory

make

(back to top)

Usage

1. To compile the project using the makefile simply execute the make command with no parameters at the root of the project.
2. To run the program execute ./bin/main.exe

Available parameters:

  -h	--help

   This parameter is optional. The default value is ''.

  -p	--path
   Set path for directory containing images. Defaults to bin/paintings directory which contains a handful of assorted artworks.
   This parameter is optional. The default value is 'bin/paintings'.

  -c	--calibration
   Path to camera calibration file
   This parameter is optional. The default value is 'bin/calibration.xml'.

  -a	--aruco
   If true, creates an ArUco marker and saves it
   This parameter is optional. The default value is '0'.

(back to top)

Roadmap

• ArUco Marker detection

  • ArUco markers can be detected by the application. Pose estimation calculated and frame axes drawn on the center of the marker.

• Placing image over ArUco Marker

  • Test image can be placed over ArUco marker. However the image sizing needs to be adjusted. The left border also sticks to the edge of the frame.

  • I was incorrectly calculating new marker corners instead of using the known marker corners that were gathered during detection. Overlay image now covers ArUco marker and rotates/translates along with it.

• I want to paste the overlay image using the first ArUco marker corner while maintaining the original size of the image. The overlay should still rotate and translate with the ArUco marker, it will just be bigger.

  • First attempt was able to match the first corners but the translation/rotation does not happen as expected. With this attempt I was using the difference between the starting points and the overlay's rows and columns. This did not account for the 3rd dimension translations.

  • Was able to get the image to overlay in a large format relative to the aruco marker and maintaining its original aspect ratio. However, it was not completely stable at certain angles.

  • Using a 560x720 frame I was able to overlay the image onto a marker. (Marker located on bookshelf)

• Application stops rendering overlay if the ArUco marker moves too quickly within the frame.

  • Using SOLVEPNP_ITERATIVE which is faster at the expense of accuracy. Provides negligible differences to the eye.
  • Storing the last known position of the overlay if detector disappears briefly.
    • This caused a lot of movement in the image and left a transparent ghost of the overlay frozen

• Allow user to cycle through images.

  • User can select a directory containing images that will be loaded into the program. Using the a and d keys will allow the user to cycle through them.

• User can display multiple images at a given time on different ArUco markers.

  • First test of displaying multiple images completed. Need to make the necessary adjustments from previous step to ensure images are the correct size.

• Clicking on an image will display some metadata in a pop up.

• Users can dynamically resize the image in real time.

• Users can assign an image to a particular marker.

See the open issues for a full list of proposed features (and known issues). -->

(back to top)

Contributing

Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.

If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with the tag "enhancement". Don't forget to give the project a star! Thanks again!

1. Fork the Project
2. Create your Feature Branch (git checkout -b feature/AmazingFeature)
3. Commit your Changes (git commit -m 'Add some AmazingFeature')
4. Push to the Branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

(back to top)

License

Distributed under the MIT License.

(back to top)

Contact

Your Name - @Golden_Sun_Kev - [email protected]

Project Link: https://github.com/Kevin-Heleodoro/Augmented-Museum

(back to top)

Acknowledgments

• Salin Boonbrahm et al., "The Use of Marker-Based Augmented Reality in Space Measurement," in Procedia Manufacturing, vol. 42, pp. 337-343, 2020, doi: 10.1016/j.promfg.2020.02.081.

• Rusnida Romli et al., "Mobile Augmented Reality (AR) Marker-based for Indoor Library Navigation," in IOP Conference Series: Materials Science and Engineering, vol. 767, 012062, 2020.

• Tommaso Tocci et al., "ArUco marker-based displacement measurement technique: uncertainty analysis," in Engineering Research Express, vol. 3, 035032, 2021.

• OpenCV Contributors, "Camera Calibration and 3D Reconstruction," 2024, OpenCV documentation.

• OpenCV Contributors, "ArucoDetector Class Reference," 2024, OpenCV documentation.

• Ikarus777, "Best Artworks of All Time," 2024, Kaggle dataset.

• Rahul Kedia, "Augmented Reality Television with ArUco Markers: OpenCV, Python, C++," 2021, Medium article.

(back to top)