Lifetime Issues in UserData #495

mrxsisyphus · 2024-12-01T14:58:03Z

mrxsisyphus
Dec 1, 2024

I'm working on creating a scraper wrapper to enable Lua to support HTML parsing. My approach is straightforward: wrap the Html and the ElementRef. Below is a simplified version of my implementation:
wrap Html:

pub struct HtmlParser(Html);

impl HtmlParser {
    pub fn with_html(html: &str) -> Self {
        HtmlParser(Html::parse_document(html))
    }
}

impl LuaUserData for HtmlParser {
    fn add_methods<M: LuaUserDataMethods<Self>>(methods: &mut M) {
        methods.add_method("select", |_, this, selector: String| {
            let selector = scraper::Selector::parse(&selector)
                .map_err(|e| anyhow!("Failed to parse selector: {}", e))?;
            let elements = this
                .0
                .select(&selector)
                .map(HtmlElement::new)
                .collect::<Vec<_>>();
            Ok(elements) //lifetime may not live long enough returning this value requires that `'1` must outlive `'2
        });
    }
}

wrap ElementRef<'a>:

pub struct HtmlElement<'lua>(ElementRef<'lua>);

impl<'lua> HtmlElement<'lua> {
    pub fn new(element: ElementRef<'lua>) -> Self {
        HtmlElement(element)
    }
}

impl<'lua> LuaUserData for HtmlElement<'lua> {
    fn add_methods<M: LuaUserDataMethods<Self>>(methods: &mut M) {
        methods.add_method("child_elements", |_, this, ()| {
            Ok(this
                .0
                .child_elements()
                .map(HtmlElement::new) //lifetime may not live long enough returning this value requires that `'lua` must outlive `'static`
                .collect::<Vec<_>>())
        });
        
    }
}

I'm encountering some lifetime issues in this implementation. I've searched through past discussions and issues and found similar problems. But I'm unsure how to apply them in this situation.

Answered by khvzak

Dec 1, 2024

You try to return an object that has reference to another object, which is not 'static, but Lua objects must be static since it does not have lifetimes.

Instead, you need to use self-referential data structures to make the object refer to itself and be 'static to meet the requirements. I would recommend ouroboros for that.
If you define HtmlElement as

#[self_referencing]
pub struct HtmlElement {
    root: Rc<Html>,
    #[borrows(root)]
    element: ElementRef<'this>
}

then you can return HtmlElement that can refer to inner (cheaply cloneable) root .

View full answer

khvzak · 2024-12-01T20:58:15Z

khvzak
Dec 1, 2024
Maintainer

You try to return an object that has reference to another object, which is not 'static, but Lua objects must be static since it does not have lifetimes.

Instead, you need to use self-referential data structures to make the object refer to itself and be 'static to meet the requirements. I would recommend ouroboros for that.
If you define HtmlElement as

#[self_referencing]
pub struct HtmlElement {
    root: Rc<Html>,
    #[borrows(root)]
    element: ElementRef<'this>
}

then you can return HtmlElement that can refer to inner (cheaply cloneable) root .

1 reply

mrxsisyphus Dec 2, 2024
Author

Thank you for your answer.
I was inspired by your answer.
I found that even when using self-referencing to allow an element to borrow the root, it still requires using another value, such as node_id.
Therefore, directly storing node_id seems to be a better choice as it can reduce the complexity caused by self-referencing.
Below is my implementation:

pub struct HtmlElement {
    html: Rc<Html>,
    node_id: NodeId,
}

impl HtmlElement {
    pub fn parse(content: &str) -> Self {
        let html = Html::parse_document(content);
        let html = Rc::new(html);
        let node_id = html.root_element().id();
        HtmlElement { html, node_id }
    }

    fn get_element_ref(&self) -> anyhow::Result<ElementRef> {
        let node = self
            .html
            .tree
            .get(self.node_id)
            .context(format!("node_id={:?} not found", self.node_id))?;
        ElementRef::wrap(node).context("node is not element")
    }

    fn make(&self, node_id: NodeId) -> Self {
        HtmlElement {
            html: self.html.clone(),
            node_id,
        }
    }

    pub fn get_selector(&self, selector: &str) -> anyhow::Result<Selector> {
        Selector::parse(selector).map_err(|e| anyhow!("parse selector={selector} err: {e}"))
    }
}

impl LuaUserData for HtmlElement {
    fn add_methods<M: LuaUserDataMethods<Self>>(methods: &mut M) {

        //text
        methods.add_method("text", |_, this, ()| {
            let element_ref = this.get_element_ref()?;
            Ok(element_ref.text().collect::<Vec<_>>().join(""))
        });
        
    
        //select
        methods.add_method("select", |_, this, selector: String| {
            let selector = this.get_selector(selector.as_str())?;
            let element_ref = this.get_element_ref()?;
            Ok(element_ref
                .select(&selector)
                .map(|elem_ref| this.make(elem_ref.id()))
                .collect::<Vec<HtmlElement>>())
        });
        //child_elements
        methods.add_method("child_elements", |_, this, ()| {
            let element_ref = this.get_element_ref()?;
            Ok(element_ref
                .child_elements()
                .map(|elem_ref| this.make(elem_ref.id()))
                .collect::<Vec<HtmlElement>>())
        });
        //...
    }
}

Although this might not be a more general approach, it works well.
I searched through past issues and discussions and found some similar examples.
using self-referencing to construct custom iterators: #300

Could you provide more suggestions on such topics?

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Lifetime Issues in UserData #495

{{title}}

Replies: 1 comment 1 reply

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Select a reply

Lifetime Issues in UserData #495

mrxsisyphus Dec 1, 2024

Replies: 1 comment · 1 reply

khvzak Dec 1, 2024 Maintainer

mrxsisyphus Dec 2, 2024 Author

mrxsisyphus
Dec 1, 2024

Replies: 1 comment 1 reply

khvzak
Dec 1, 2024
Maintainer

mrxsisyphus Dec 2, 2024
Author