PENGY

UN's Sustainable Development

Goal 11. Sustainable Cities and Communities

Problem

1. Not Paying Attention to Fire Risks: A lot of people don’t realize how everyday actions and oversights can lead to serious fire hazards. Things like plugging too many devices into one outlet, not storing things safely, or forgetting to check if fire safety gear works properly are common mistakes. Many think, "A fire won’t happen to me," and end up ignoring basic safety steps.

2. Radio-like one-way education: The current approach to fire safety education often delivers information that is not directly applicable to the specific environments in which we live or work. This lack of personalized and relevant content makes it difficult for individuals to truly grasp the importance of fire safety or to envision how to implement these practices in their daily lives. To address this issue, there's a pressing need to shift from a one-size-fits-all method of instruction to a more interactive and engaging educational model.

3. Not Looking for Information on Fire Safety: People generally don’t make an effort to learn fire safety knowledge by their own will. Since many think a fire is unlikely to happen to them, they don’t see the point in spending time on this topic. Even when helpful tips and information are easy to find, they're often ignored. This attitude stops important safety information from getting to the people who need it most and makes them less prepared for fire risks around them.

Solution

Our Solution addresses the issue of underestimating fire risks due to a lack of fire safety awareness and preparedness. By transforming the perception of fire from a distant to a daily concern, we aim to heighten awareness of fire hazards and provide knowledge for appropriate responses.

Key features:

1. Management of Fire Hazards in Personal Spaces

2. Fire Safety Education through diverse content including news, YouTube videos, fire prevention tips, and academic theses.

3. Content Generation: These contents are semi-automatically generated through crawling and open-source APIs.

4. Customized Advice: Tailored advice for fire management specific to different spaces, generated by Gemini Vision Pro.

5. VertexAI with Grounding: Added for verifying response reliability.

6. Temperature-Based Risk Assessment: Spaces' risk levels are now measurable via thermometers, reflected differently on Google Maps for easy visualization.

7. Daily Fire Safety Tips: Using google batch, delivered via push notifications (Firebase Cloud Messaging) tailored to user's registered spaces.

8. Fire Safety Quizzes: To test and improve knowledge, with enhanced UX during waiting time for Gemini's response.

9. Safety Checklist per space.

10. Real-time Object Detection for Fire Risk Assessment: YOLO v8 ML model trained on top 7 fire hazard categories for personalized recommendations, integrated with camera feed for real-time detection and suggestion of fire risk objects.

How to run

[Front-End] Click to download a releaed apk. To install this, you need to able downloading an app from unknown sources.

or

using flutter

1. Clone this project

    git clone https://github.com/GDSC-CAU/Pengy-FE.git

2. Set .env in root project

  MAP_KEY=*Put your Google Maps API KEY*

3. Set local.properties in root project/android/local.properties

  MAP_KEY=*Put your Google Maps API KEY*

4. Run with Android Studio

[Back-End] click to redirect to Back-End repository.

1. Clone this project

    git clone https://github.com/GDSC-CAU/Pengy-BE.git

2. Create a Python virtual environmen

    python -m venv venv
    ./venv/scripts/activate

3. Install dependencies and start the server.

    pip install -r requirements.txt
    python manage.py runserver

[AI] click to redirect to AI repository.

1. Clone this project

    git clone https://github.com/GDSC-CAU/Pengy-AI.git

2. Create a Python virtual environmen

    python -m venv venv
    ./venv/scripts/activate

3. Install dependencies

    pip install -r requirements.txt

4. if you wanna train your model, refer to jupyter

    cd jupyter

5. for object detection with webcam

    python detect.py your.model

6. export your model to tflite

    python export.py your.model tflite

System Architecture

Used open source

• yolo v8: https://github.com/ultralytics/ultralytics
• scholarly: https://github.com/scholarly-python-package/scholarly

Google Products

• Flutter
• Firebase
• TensorFlow Lite
• Google Cloud Platform
• Google Maps API
• Google Login API
• YouTube API
• Gemini API
• Vertex AI

Screens

MainPage

BeforeCamera	Loading Analysis Result	Space Analysis Result

Space Check List	Fire Hazard Info	Space Analysis Result

Prompt that we wrote

def construct_prompt(image_url):
    image_url = Image.open(image_url)
    """Constructs the detailed prompt for the AI model based on the image URL."""
    task_details = (
        "Let's think step by step. Demonstrate deep expertise in fire safety. "
        "ROLE: You are a fire investigation expert; "
        "YOUR TASK IS: "
        "0. Describe the place or object in the photo in ONE PHRASE"
        """1. Specify the degree of fire vulnerability out of 100, 
            Consider the following factors to assess the degree of fire vulnerability: 
            i) Visibility of combustible materials: Assess the amount and type of combustible materials visible in the photo. For example, if stacks of paper, cardboard boxes, furniture (made of wood), etc. are visible in the picture, this can increase the risk of fire.;
            ii) Condition of electrical equipment and wiring: Identify the visible condition of electrical equipment and potential hazards. For example, old or damaged electrical outlets, misplaced wires, and multi-taps that appear to be overloaded.;
            iii) Fire alarm and safety equipment: Verify the presence and condition of fire alarms, sprinklers, fire extinguishers, etc. Assess whether they are visible in photos and in easily accessible locations.;
            iv) Density and organization of the space: Assess how well organized the space is and how crowded it is. Excessive stacking of items or obstructions can increase the risk by blocking escape routes in the event of a fire.;
            v) Visibility of ventilation: Assess the size and number of vents or windows. Well-designed ventilation is important for quickly removing smoke and heat in the event of a fire.;
            Specify the degree of fire vulnerability out of 100. 
            ONLY NUMERIC VALUES ARE ACCEPTED.
        """
        "2. Identify the fire hazards in the photo and return the specific location of it, "
        "3. Advise on measures to mitigate the fire risks according to the 'priority', "
        "4. List 'fewer than three objects' that are likely to cause a fire in the photo, in one sentence without explanations. "
        "If the objects are similar, use a representative term. E.g., refrigerator, washing machine, air conditioner..."
        "Ensure : that your answer is unbiased, detailed and avoids relying on stereotypes. Demonstrate deep expertise in fire safety."
        "NOTE : I will give you $12,000 tip for a better solution."
    )
    response_format = (
        "JSON RESPONSE FORMAT: {"
            "\"Description\": \"<phrase>\", "
            "\"Degree of Fire Danger (out of 100)\": \"<number>\", "
            "\"Identified Fire Hazards\": \"<object1>\", \"<object2>\", ..."
            "\"Mitigation Measures\": \"<text>\", "
            "\"Additional Recommendation\": \"<text>\", "
            "\"Top 3 fire Hazards objects\": \"<object1>\, \"<object2>\", ..."
        "}"
    )
    full_prompt = f"\"prompt:{{ {task_details} {response_format} }}\""
    prompt_parts = [image_url, "\n\n", full_prompt, "\n\n\n"]
    return prompt_parts

Team

Janghyeon Park (Leader)	Sunbin Do	Byeori Moon	Junhyung Park
Back-End/AI	AI/Back-End	Front-End/Design	Front-End/AI
park-janghyeon	typingmistake	byeori-moon	DogJHDOG